Sep 10 15:21:15 corosync [MAIN  ] Corosync Cluster Engine ('1.4.3'): started and ready to provide service.
Sep 10 15:21:15 corosync [MAIN  ] Corosync built-in features: nss
Sep 10 15:21:15 corosync [MAIN  ] Successfully read config from /etc/cluster/cluster.conf
Sep 10 15:21:15 corosync [MAIN  ] Successfully parsed cman config
Sep 10 15:21:15 corosync [MAIN  ] Successfully configured openais services to load
Sep 10 15:21:15 corosync [TOTEM ] Initializing transport (UDP/IP Unicast).
Sep 10 15:21:15 corosync [TOTEM ] Initializing transmit/receive security: libtomcrypt SOBER128/SHA1HMAC (mode 0).
Sep 10 15:21:15 corosync [TOTEM ] The network interface [192.168.1.199] is now up.
Sep 10 15:21:15 corosync [QUORUM] Using quorum provider quorum_cman
Sep 10 15:21:15 corosync [SERV  ] Service engine loaded: corosync cluster quorum service v0.1
Sep 10 15:21:15 corosync [CMAN  ] CMAN 1341928020 (built Jul 10 2012 16:47:17) started
Sep 10 15:21:15 corosync [SERV  ] Service engine loaded: corosync CMAN membership service 2.90
Sep 10 15:21:15 corosync [SERV  ] Service engine loaded: openais cluster membership service B.01.01
Sep 10 15:21:15 corosync [SERV  ] Service engine loaded: openais event service B.01.01
Sep 10 15:21:15 corosync [SERV  ] Service engine loaded: openais checkpoint service B.01.01
Sep 10 15:21:15 corosync [SERV  ] Service engine loaded: openais message service B.03.01
Sep 10 15:21:15 corosync [SERV  ] Service engine loaded: openais distributed locking service B.03.01
Sep 10 15:21:15 corosync [SERV  ] Service engine loaded: openais timer service A.01.01
Sep 10 15:21:15 corosync [SERV  ] Service engine loaded: corosync extended virtual synchrony service
Sep 10 15:21:15 corosync [SERV  ] Service engine loaded: corosync configuration service
Sep 10 15:21:15 corosync [SERV  ] Service engine loaded: corosync cluster closed process group service v1.01
Sep 10 15:21:15 corosync [SERV  ] Service engine loaded: corosync cluster config database access v1.01
Sep 10 15:21:15 corosync [SERV  ] Service engine loaded: corosync profile loading service
Sep 10 15:21:15 corosync [QUORUM] Using quorum provider quorum_cman
Sep 10 15:21:15 corosync [SERV  ] Service engine loaded: corosync cluster quorum service v0.1
Sep 10 15:21:15 corosync [MAIN  ] Compatibility mode set to whitetank.  Using V1 and V2 of the synchronization engine.
Sep 10 15:21:15 corosync [CLM   ] CLM CONFIGURATION CHANGE
Sep 10 15:21:15 corosync [CLM   ] New Configuration:
Sep 10 15:21:15 corosync [CLM   ] Members Left:
Sep 10 15:21:15 corosync [CLM   ] Members Joined:
Sep 10 15:21:15 corosync [CLM   ] CLM CONFIGURATION CHANGE
Sep 10 15:21:15 corosync [CLM   ] New Configuration:
Sep 10 15:21:15 corosync [CLM   ] 	r(0) ip(192.168.1.199) 
Sep 10 15:21:15 corosync [CLM   ] Members Left:
Sep 10 15:21:15 corosync [CLM   ] Members Joined:
Sep 10 15:21:15 corosync [CLM   ] 	r(0) ip(192.168.1.199) 
Sep 10 15:21:15 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed.
Sep 10 15:21:15 corosync [CLM   ] CLM CONFIGURATION CHANGE
Sep 10 15:21:15 corosync [CLM   ] New Configuration:
Sep 10 15:21:15 corosync [CLM   ] 	r(0) ip(192.168.1.199) 
Sep 10 15:21:15 corosync [CLM   ] Members Left:
Sep 10 15:21:15 corosync [CLM   ] Members Joined:
Sep 10 15:21:15 corosync [CLM   ] CLM CONFIGURATION CHANGE
Sep 10 15:21:15 corosync [CLM   ] New Configuration:
Sep 10 15:21:15 corosync [CLM   ] 	r(0) ip(192.168.1.199) 
Sep 10 15:21:15 corosync [CLM   ] 	r(0) ip(192.168.1.200) 
Sep 10 15:21:15 corosync [CLM   ] Members Left:
Sep 10 15:21:15 corosync [CLM   ] Members Joined:
Sep 10 15:21:15 corosync [CLM   ] 	r(0) ip(192.168.1.200) 
Sep 10 15:21:15 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed.
Sep 10 15:21:15 corosync [CMAN  ] quorum regained, resuming activity
Sep 10 15:21:15 corosync [QUORUM] This node is within the primary component and will provide service.
Sep 10 15:21:15 corosync [QUORUM] Members[1]: 2
Sep 10 15:21:15 corosync [QUORUM] Members[1]: 2
Sep 10 15:21:15 corosync [QUORUM] Members[2]: 1 2
Sep 10 15:21:15 corosync [QUORUM] Members[2]: 1 2
Sep 10 15:21:15 corosync [CPG   ] chosen downlist: sender r(0) ip(192.168.1.200) ; members(old:1 left:0)
Sep 10 15:21:15 corosync [MAIN  ] Completed service synchronization, ready to provide service.
Set r/w permissions for uid=666, gid=0 on /var/log/cluster/corosync.log
Sep 10 15:21:19 Cluster-Server-1 pacemakerd: [48700]: info: crm_log_init_worker: Changed active directory to /var/lib/heartbeat/cores/root
Sep 10 15:21:19 Cluster-Server-1 pacemakerd: [48700]: notice: main: Starting Pacemaker 1.1.7 (Build: ee0730e13d124c3d58f00016c3376a1de5323cff):   corosync-plugin cman

Sep 10 15:21:19 Cluster-Server-1 pacemakerd: [48700]: info: main: Maximum core file size is: 18446744073709551615
Sep 10 15:21:19 Cluster-Server-1 pacemakerd: [48700]: debug: cluster_connect_cfg: Our nodeid: 1
Sep 10 15:21:19 Cluster-Server-1 pacemakerd: [48700]: debug: cluster_connect_cfg: Adding fd=6 to mainloop
Sep 10 15:21:19 Cluster-Server-1 pacemakerd: [48700]: debug: cluster_connect_cpg: Our nodeid: 1
Sep 10 15:21:19 Cluster-Server-1 pacemakerd: [48700]: debug: cluster_connect_cpg: Adding fd=7 to mainloop
Sep 10 15:21:19 Cluster-Server-1 pacemakerd: [48700]: info: get_local_node_name: Using CMAN node name: Cluster-Server-1
Sep 10 15:21:19 Cluster-Server-1 pacemakerd: [48700]: notice: update_node_processes: 0x17e5e00 Node 1 now known as Cluster-Server-1, was: 
Sep 10 15:21:19 Cluster-Server-1 pacemakerd: [48700]: debug: update_node_processes: Node Cluster-Server-1 now has process list: 00000000000000000000000000000002 (was 00000000000000000000000000000000)
Sep 10 15:21:19 Cluster-Server-1 pacemakerd: [48700]: info: start_child: Forked child 48709 for process cib
Sep 10 15:21:19 Cluster-Server-1 pacemakerd: [48700]: debug: update_node_processes: Node Cluster-Server-1 now has process list: 00000000000000000000000000000102 (was 00000000000000000000000000000002)
Sep 10 15:21:19 Cluster-Server-1 pacemakerd: [48700]: info: start_child: Forked child 48710 for process stonith-ng
Sep 10 15:21:19 Cluster-Server-1 pacemakerd: [48700]: debug: update_node_processes: Node Cluster-Server-1 now has process list: 00000000000000000000000000100102 (was 00000000000000000000000000000102)
Sep 10 15:21:19 Cluster-Server-1 pacemakerd: [48700]: info: start_child: Forked child 48712 for process lrmd
Sep 10 15:21:19 Cluster-Server-1 pacemakerd: [48700]: debug: update_node_processes: Node Cluster-Server-1 now has process list: 00000000000000000000000000100112 (was 00000000000000000000000000100102)
Sep 10 15:21:19 Cluster-Server-1 pacemakerd: [48700]: info: start_child: Forked child 48713 for process attrd
Sep 10 15:21:19 Cluster-Server-1 pacemakerd: [48700]: debug: update_node_processes: Node Cluster-Server-1 now has process list: 00000000000000000000000000101112 (was 00000000000000000000000000100112)
Sep 10 15:21:19 Cluster-Server-1 pacemakerd: [48700]: info: start_child: Forked child 48714 for process pengine
Sep 10 15:21:19 Cluster-Server-1 pacemakerd: [48700]: debug: update_node_processes: Node Cluster-Server-1 now has process list: 00000000000000000000000000111112 (was 00000000000000000000000000101112)
Sep 10 15:21:19 Cluster-Server-1 stonith-ng: [48710]: info: Invoked: /usr/libexec/pacemaker/stonithd 
Sep 10 15:21:19 Cluster-Server-1 pacemakerd: [48700]: info: start_child: Forked child 48715 for process crmd
Sep 10 15:21:19 Cluster-Server-1 pacemakerd: [48700]: debug: update_node_processes: Node Cluster-Server-1 now has process list: 00000000000000000000000000111312 (was 00000000000000000000000000111112)
Sep 10 15:21:19 Cluster-Server-1 pacemakerd: [48700]: info: main: Starting mainloop
Sep 10 15:21:19 Cluster-Server-1 pacemakerd: [48700]: notice: update_node_processes: 0x17ea940 Node 2 now known as Cluster-Server-2, was: 
Sep 10 15:21:19 Cluster-Server-1 pacemakerd: [48700]: debug: update_node_processes: Node Cluster-Server-2 now has process list: 00000000000000000000000000111312 (was 00000000000000000000000000000000)
Sep 10 15:21:19 Cluster-Server-1 lrmd: [48712]: info: enabling coredumps
Sep 10 15:21:19 Cluster-Server-1 lrmd: [48712]: debug: main: run the loop...
Sep 10 15:21:19 Cluster-Server-1 lrmd: [48712]: info: Started.
Sep 10 15:21:19 Cluster-Server-1 crmd: [48715]: info: Invoked: /usr/libexec/pacemaker/crmd 
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: info: crm_log_init_worker: Changed active directory to /var/lib/heartbeat/cores/hacluster
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: info: retrieveCib: Reading cluster configuration from: /var/lib/heartbeat/crm/cib.xml (digest: /var/lib/heartbeat/crm/cib.xml.sig)
Sep 10 15:21:19 Cluster-Server-1 stonith-ng: [48710]: info: crm_log_init_worker: Changed active directory to /var/lib/heartbeat/cores/root
Sep 10 15:21:19 Cluster-Server-1 stonith-ng: [48710]: info: get_cluster_type: Cluster type is: 'cman'
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: WARN: retrieveCib: Cluster configuration not found: /var/lib/heartbeat/crm/cib.xml
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: WARN: readCibXmlFile: Primary configuration corrupt or unusable, trying backup...
Sep 10 15:21:19 Cluster-Server-1 stonith-ng: [48710]: notice: crm_cluster_connect: Connecting to cluster infrastructure: cman
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: debug: get_last_sequence: Series file /var/lib/heartbeat/crm/cib.last does not exist
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: debug: readCibXmlFile: Backup file /var/lib/heartbeat/crm/cib-99.raw not found
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: WARN: readCibXmlFile: Continuing with an empty configuration.
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: debug: readCibXmlFile: [on-disk] <cib epoch="0" num_updates="0" admin_epoch="0" validate-with="pacemaker-1.2" >
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: debug: readCibXmlFile: [on-disk]   <configuration >
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: debug: readCibXmlFile: [on-disk]     <crm_config />
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: debug: readCibXmlFile: [on-disk]     <nodes />
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: debug: readCibXmlFile: [on-disk]     <resources />
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: debug: readCibXmlFile: [on-disk]     <constraints />
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: debug: readCibXmlFile: [on-disk]   </configuration>
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: debug: readCibXmlFile: [on-disk]   <status />
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: debug: readCibXmlFile: [on-disk] </cib>
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: info: validate_with_relaxng: Creating RNG parser context
Sep 10 15:21:19 Cluster-Server-1 attrd: [48713]: info: Invoked: /usr/libexec/pacemaker/attrd 
Sep 10 15:21:19 Cluster-Server-1 attrd: [48713]: info: crm_log_init_worker: Changed active directory to /var/lib/heartbeat/cores/hacluster
Sep 10 15:21:19 Cluster-Server-1 attrd: [48713]: info: main: Starting up
Sep 10 15:21:19 Cluster-Server-1 attrd: [48713]: info: get_cluster_type: Cluster type is: 'cman'
Sep 10 15:21:19 Cluster-Server-1 attrd: [48713]: notice: crm_cluster_connect: Connecting to cluster infrastructure: cman
Sep 10 15:21:19 Cluster-Server-1 crmd: [48715]: info: crm_log_init_worker: Changed active directory to /var/lib/heartbeat/cores/root
Sep 10 15:21:19 Cluster-Server-1 crmd: [48715]: notice: main: CRM Git Version: ee0730e13d124c3d58f00016c3376a1de5323cff

Sep 10 15:21:19 Cluster-Server-1 crmd: [48715]: debug: crmd_init: Starting crmd
Sep 10 15:21:19 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_STARTUP: [ state=S_STARTING cause=C_STARTUP origin=crmd_init ]
Sep 10 15:21:19 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_LOG   
Sep 10 15:21:19 Cluster-Server-1 crmd: [48715]: debug: do_log: FSA: Input I_STARTUP from crmd_init() received in state S_STARTING
Sep 10 15:21:19 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_STARTUP
Sep 10 15:21:19 Cluster-Server-1 crmd: [48715]: debug: do_startup: Registering Signal Handlers
Sep 10 15:21:19 Cluster-Server-1 crmd: [48715]: debug: do_startup: Creating CIB and LRM objects
Sep 10 15:21:19 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_CIB_START
Sep 10 15:21:19 Cluster-Server-1 crmd: [48715]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:21:19 Cluster-Server-1 crmd: [48715]: debug: init_client_ipc_comms_nodispatch: Could not init comms on: /var/run/crm/cib_rw
Sep 10 15:21:19 Cluster-Server-1 crmd: [48715]: debug: cib_native_signon_raw: Connection to command channel failed
Sep 10 15:21:19 Cluster-Server-1 crmd: [48715]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:21:19 Cluster-Server-1 pengine: [48714]: info: Invoked: /usr/libexec/pacemaker/pengine 
Sep 10 15:21:19 Cluster-Server-1 pengine: [48714]: info: crm_log_init_worker: Changed active directory to /var/lib/heartbeat/cores/hacluster
Sep 10 15:21:19 Cluster-Server-1 pengine: [48714]: debug: main: Checking for old instances of pengine
Sep 10 15:21:19 Cluster-Server-1 crmd: [48715]: debug: init_client_ipc_comms_nodispatch: Could not init comms on: /var/run/crm/cib_callback
Sep 10 15:21:19 Cluster-Server-1 pengine: [48714]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/pengine
Sep 10 15:21:19 Cluster-Server-1 pengine: [48714]: debug: init_client_ipc_comms_nodispatch: Could not init comms on: /var/run/crm/pengine
Sep 10 15:21:19 Cluster-Server-1 pengine: [48714]: debug: main: Init server comms
Sep 10 15:21:19 Cluster-Server-1 pengine: [48714]: info: main: Starting pengine
Sep 10 15:21:19 Cluster-Server-1 crmd: [48715]: debug: cib_native_signon_raw: Connection to callback channel failed
Sep 10 15:21:19 Cluster-Server-1 crmd: [48715]: debug: cib_native_signon_raw: Connection to CIB failed: connection failed
Sep 10 15:21:19 Cluster-Server-1 crmd: [48715]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:21:19 Cluster-Server-1 stonith-ng: [48710]: debug: init_cpg_connection: Adding fd=5 to mainloop
Sep 10 15:21:19 Cluster-Server-1 stonith-ng: [48710]: info: get_local_node_name: Using CMAN node name: Cluster-Server-1
Sep 10 15:21:19 Cluster-Server-1 stonith-ng: [48710]: info: init_ais_connection_once: Connection to 'cman': established
Sep 10 15:21:19 Cluster-Server-1 stonith-ng: [48710]: debug: crm_new_peer: Creating entry for node Cluster-Server-1/1
Sep 10 15:21:19 Cluster-Server-1 stonith-ng: [48710]: info: crm_new_peer: Node Cluster-Server-1 now has id: 1
Sep 10 15:21:19 Cluster-Server-1 stonith-ng: [48710]: info: crm_new_peer: Node 1 is now known as Cluster-Server-1
Sep 10 15:21:19 Cluster-Server-1 stonith-ng: [48710]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/pcmk
Sep 10 15:21:19 Cluster-Server-1 pacemakerd: [48700]: debug: pcmk_client_connect: Channel 0x17eb1a0 connected: 1 children
Sep 10 15:21:19 Cluster-Server-1 attrd: [48713]: debug: init_cpg_connection: Adding fd=5 to mainloop
Sep 10 15:21:19 Cluster-Server-1 attrd: [48713]: info: get_local_node_name: Using CMAN node name: Cluster-Server-1
Sep 10 15:21:19 Cluster-Server-1 attrd: [48713]: info: init_ais_connection_once: Connection to 'cman': established
Sep 10 15:21:19 Cluster-Server-1 attrd: [48713]: debug: crm_new_peer: Creating entry for node Cluster-Server-1/1
Sep 10 15:21:19 Cluster-Server-1 attrd: [48713]: info: crm_new_peer: Node Cluster-Server-1 now has id: 1
Sep 10 15:21:19 Cluster-Server-1 attrd: [48713]: info: crm_new_peer: Node 1 is now known as Cluster-Server-1
Sep 10 15:21:19 Cluster-Server-1 attrd: [48713]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/pcmk
Sep 10 15:21:19 Cluster-Server-1 attrd: [48713]: info: main: Cluster connection active
Sep 10 15:21:19 Cluster-Server-1 attrd: [48713]: info: main: Accepting attribute updates
Sep 10 15:21:19 Cluster-Server-1 attrd: [48713]: notice: main: Starting mainloop...
Sep 10 15:21:19 Cluster-Server-1 pacemakerd: [48700]: debug: pcmk_client_connect: Channel 0x17ec670 connected: 2 children
Sep 10 15:21:19 Cluster-Server-1 attrd: [48713]: debug: pcmk_cpg_membership: Member[0] 1 
Sep 10 15:21:19 Cluster-Server-1 attrd: [48713]: debug: pcmk_cpg_membership: Member[1] 2 
Sep 10 15:21:19 Cluster-Server-1 attrd: [48713]: debug: crm_update_peer: Node Cluster-Server-1: id=1 seen=0 proc=00000000000000000000000000111312 (new)
Sep 10 15:21:19 Cluster-Server-1 attrd: [48713]: debug: crm_new_peer: Creating entry for node Cluster-Server-2/2
Sep 10 15:21:19 Cluster-Server-1 attrd: [48713]: info: crm_new_peer: Node Cluster-Server-2 now has id: 2
Sep 10 15:21:19 Cluster-Server-1 attrd: [48713]: info: crm_new_peer: Node 2 is now known as Cluster-Server-2
Sep 10 15:21:19 Cluster-Server-1 stonith-ng: [48710]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:21:19 Cluster-Server-1 stonith-ng: [48710]: debug: init_client_ipc_comms_nodispatch: Could not init comms on: /var/run/crm/cib_rw
Sep 10 15:21:19 Cluster-Server-1 stonith-ng: [48710]: debug: cib_native_signon_raw: Connection to command channel failed
Sep 10 15:21:19 Cluster-Server-1 stonith-ng: [48710]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:21:19 Cluster-Server-1 stonith-ng: [48710]: debug: init_client_ipc_comms_nodispatch: Could not init comms on: /var/run/crm/cib_callback
Sep 10 15:21:19 Cluster-Server-1 stonith-ng: [48710]: debug: cib_native_signon_raw: Connection to callback channel failed
Sep 10 15:21:19 Cluster-Server-1 stonith-ng: [48710]: debug: cib_native_signon_raw: Connection to CIB failed: connection failed
Sep 10 15:21:19 Cluster-Server-1 stonith-ng: [48710]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for start op
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: info: startCib: CIB Initialization completed successfully
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: info: get_cluster_type: Cluster type is: 'cman'
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: notice: crm_cluster_connect: Connecting to cluster infrastructure: cman
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: debug: init_cpg_connection: Adding fd=5 to mainloop
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: info: get_local_node_name: Using CMAN node name: Cluster-Server-1
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: info: init_ais_connection_once: Connection to 'cman': established
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: debug: crm_new_peer: Creating entry for node Cluster-Server-1/1
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: info: crm_new_peer: Node Cluster-Server-1 now has id: 1
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: info: crm_new_peer: Node 1 is now known as Cluster-Server-1
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/pcmk
Sep 10 15:21:19 Cluster-Server-1 pacemakerd: [48700]: debug: pcmk_client_connect: Channel 0x17ec8e0 connected: 3 children
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: info: cib_init: Starting cib mainloop
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: debug: pcmk_cpg_membership: Member[0] 1 
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: debug: pcmk_cpg_membership: Member[1] 2 
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: debug: crm_update_peer: Node Cluster-Server-1: id=1 seen=0 proc=00000000000000000000000000111312 (new)
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: debug: crm_new_peer: Creating entry for node Cluster-Server-2/2
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: info: crm_new_peer: Node Cluster-Server-2 now has id: 2
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: info: crm_new_peer: Node 2 is now known as Cluster-Server-2
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: debug: Forking temp process write_cib_contents
Sep 10 15:21:19 Cluster-Server-1 cib: [48709]: info: Managed write_cib_contents process 48725 exited with return code 0.
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: info: do_cib_control: CIB connection established
Sep 10 15:21:20 Cluster-Server-1 cib: [48709]: debug: cib_common_callback_worker: Setting cib_refresh_notify callbacks for 48715 (e00c5ff0-ca9f-4fba-a6df-4b13f36ff633): on
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_HA_CONNECT
Sep 10 15:21:20 Cluster-Server-1 cib: [48709]: debug: cib_common_callback_worker: Setting cib_diff_notify callbacks for 48715 (e00c5ff0-ca9f-4fba-a6df-4b13f36ff633): on
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: info: get_cluster_type: Cluster type is: 'cman'
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: notice: crm_cluster_connect: Connecting to cluster infrastructure: cman
Sep 10 15:21:20 Cluster-Server-1 stonith-ng: [48710]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:21:20 Cluster-Server-1 stonith-ng: [48710]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:21:20 Cluster-Server-1 stonith-ng: [48710]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:21:20 Cluster-Server-1 stonith-ng: [48710]: notice: setup_cib: Watching for stonith topology changes
Sep 10 15:21:20 Cluster-Server-1 stonith-ng: [48710]: info: main: Starting stonith-ng mainloop
Sep 10 15:21:20 Cluster-Server-1 stonith-ng: [48710]: debug: pcmk_cpg_membership: Member[0] 1 
Sep 10 15:21:20 Cluster-Server-1 stonith-ng: [48710]: debug: pcmk_cpg_membership: Member[1] 2 
Sep 10 15:21:20 Cluster-Server-1 stonith-ng: [48710]: debug: crm_update_peer: Node Cluster-Server-1: id=1 seen=0 proc=00000000000000000000000000111312 (new)
Sep 10 15:21:20 Cluster-Server-1 stonith-ng: [48710]: debug: crm_new_peer: Creating entry for node Cluster-Server-2/2
Sep 10 15:21:20 Cluster-Server-1 stonith-ng: [48710]: info: crm_new_peer: Node Cluster-Server-2 now has id: 2
Sep 10 15:21:20 Cluster-Server-1 stonith-ng: [48710]: info: crm_new_peer: Node 2 is now known as Cluster-Server-2
Sep 10 15:21:20 Cluster-Server-1 cib: [48709]: debug: cib_common_callback_worker: Setting cib_diff_notify callbacks for 48710 (928f7513-3a99-4c91-aada-ea04881ab5f4): on
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: init_cpg_connection: Adding fd=7 to mainloop
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: info: get_local_node_name: Using CMAN node name: Cluster-Server-1
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: info: init_ais_connection_once: Connection to 'cman': established
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: crm_new_peer: Creating entry for node Cluster-Server-1/1
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: info: crm_new_peer: Node Cluster-Server-1 now has id: 1
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: info: crm_new_peer: Node 1 is now known as Cluster-Server-1
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: info: ais_status_callback: status: Cluster-Server-1 is now unknown
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/pcmk
Sep 10 15:21:20 Cluster-Server-1 pacemakerd: [48700]: debug: pcmk_client_connect: Channel 0x17ec9f0 connected: 4 children
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: info: init_cman_connection: Configuring Pacemaker to obtain quorum from cman
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: notice: cman_event_callback: Membership 312: quorum acquired
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: info: ais_status_callback: status: Cluster-Server-1 is now member (was unknown)
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: info: crm_update_peer: Node Cluster-Server-1: id=1 state=member (new) addr=(null) votes=0 born=312 seen=312 proc=00000000000000000000000000000000
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: crm_new_peer: Creating entry for node Cluster-Server-2/2
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: info: crm_new_peer: Node Cluster-Server-2 now has id: 2
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: info: crm_new_peer: Node 2 is now known as Cluster-Server-2
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: info: ais_status_callback: status: Cluster-Server-2 is now unknown
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: info: ais_status_callback: status: Cluster-Server-2 is now member (was unknown)
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: info: crm_update_peer: Node Cluster-Server-2: id=2 state=member (new) addr=(null) votes=0 born=312 seen=312 proc=00000000000000000000000000000000
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: post_cache_update: Updated cache after membership event 312.
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: post_cache_update: post_cache_update added action A_ELECTION_CHECK to the FSA
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: init_cman_connection: Adding fd=9 to mainloop
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: info: do_ha_control: Connected to the cluster
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_READCONFIG
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_LRM_CONNECT
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: do_lrm_control: Connecting to the LRM
Sep 10 15:21:20 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client crmd [48715] registered
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: do_lrm_control: LRM connection established
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_CCM_CONNECT
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_STARTED
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: info: do_started: Delaying start, Config not read (0000000000000040)
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: register_fsa_input_adv: Stalling the FSA pending further input: cause=C_FSA_INTERNAL
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Exiting the FSA: queue=0, fsa_actions=0x200000002, stalled=true
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_STARTED
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: info: do_started: Delaying start, Config not read (0000000000000040)
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: register_fsa_input_adv: Stalling the FSA pending further input: cause=C_FSA_INTERNAL
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Exiting the FSA: queue=0, fsa_actions=0x200000002, stalled=true
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: config_query_callback: Call 3 : Parsing CIB options
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: config_query_callback: Shutdown escalation occurs after: 1200000ms
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: config_query_callback: Checking for expired actions every 900000ms
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: pcmk_cpg_membership: Member[0] 1 
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: pcmk_cpg_membership: Member[1] 2 
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: notice: crmd_peer_update: Status update: Client Cluster-Server-1/crmd now has status [online] (DC=<null>)
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: crm_update_peer: Node Cluster-Server-1: id=1 seen=312 proc=00000000000000000000000000111312 (new)
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: notice: crmd_peer_update: Status update: Client Cluster-Server-2/crmd now has status [online] (DC=<null>)
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: crm_update_peer: Node Cluster-Server-2: id=2 seen=312 proc=00000000000000000000000000111312 (new)
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_STARTED
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: do_started: Init server comms
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: notice: do_started: The local CRM is operational
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_CHECK
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: do_election_check: Ignore election check: we not in an election
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_PENDING: [ state=S_STARTING cause=C_FSA_INTERNAL origin=do_started ]
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_LOG   
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: do_log: FSA: Input I_PENDING from do_started() received in state S_STARTING
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: notice: do_state_transition: State transition S_STARTING -> S_PENDING [ input=I_PENDING cause=C_FSA_INTERNAL origin=do_started ]
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_INTEGRATE_TIMER_STOP
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_FINALIZE_TIMER_STOP
Sep 10 15:21:20 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_CL_JOIN_QUERY
Sep 10 15:21:21 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_query: Querying for a DC
Sep 10 15:21:21 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_START
Sep 10 15:21:21 Cluster-Server-1 crmd: [48715]: debug: crm_timer_start: Started Election Trigger (I_DC_TIMEOUT:20000ms), src=14
Sep 10 15:21:21 Cluster-Server-1 crmd: [48715]: debug: te_connect_stonith: Attempting connection to fencing daemon...
Sep 10 15:21:22 Cluster-Server-1 crmd: [48715]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/st_command
Sep 10 15:21:22 Cluster-Server-1 crmd: [48715]: debug: get_stonith_token: Obtained registration token: 9694a93b-1cd0-4228-a8b3-b6ead5d6ff79
Sep 10 15:21:22 Cluster-Server-1 crmd: [48715]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/st_callback
Sep 10 15:21:22 Cluster-Server-1 crmd: [48715]: debug: get_stonith_token: Obtained registration token: f0678716-b283-4a01-8e60-13b1b327f16e
Sep 10 15:21:22 Cluster-Server-1 crmd: [48715]: debug: stonith_api_signon: Connection to STONITH successful
Sep 10 15:21:22 Cluster-Server-1 stonith-ng: [48710]: debug: stonith_command: Processing register from crmd (               0)
Sep 10 15:21:22 Cluster-Server-1 stonith-ng: [48710]: debug: stonith_command: Processing st_notify from 48715 (               0)
Sep 10 15:21:22 Cluster-Server-1 stonith-ng: [48710]: debug: stonith_command: Setting st_notify_disconnect callbacks for 48715 (f0678716-b283-4a01-8e60-13b1b327f16e): ON
Sep 10 15:21:22 Cluster-Server-1 stonith-ng: [48710]: debug: stonith_command: Processing st_notify from 48715 (               0)
Sep 10 15:21:22 Cluster-Server-1 stonith-ng: [48710]: debug: stonith_command: Setting st_fence callbacks for 48715 (f0678716-b283-4a01-8e60-13b1b327f16e): ON
Sep 10 15:21:24 Cluster-Server-1 attrd: [48713]: debug: cib_connect: CIB signon attempt 1
Sep 10 15:21:24 Cluster-Server-1 attrd: [48713]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:21:24 Cluster-Server-1 attrd: [48713]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:21:24 Cluster-Server-1 attrd: [48713]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:21:24 Cluster-Server-1 attrd: [48713]: info: cib_connect: Connected to the CIB after 1 signon attempts
Sep 10 15:21:24 Cluster-Server-1 attrd: [48713]: info: cib_connect: Sending full refresh
Sep 10 15:21:24 Cluster-Server-1 cib: [48709]: debug: cib_common_callback_worker: Setting cib_refresh_notify callbacks for 48713 (e8e0717f-6a38-4c76-a1f9-27b7b5c96e9c): on
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_COUNT
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_election_count_vote: Created voted hash
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: crm_uptime: Current CPU usage is: 0s, 10000us
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: info: do_election_count_vote: Election 2 (owner: Cluster-Server-2) pass: vote from Cluster-Server-2 (Host name)
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_CHECK
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_election_check: Ignore election check: we not in an election
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_ELECTION: [ state=S_PENDING cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: notice: do_state_transition: State transition S_PENDING -> S_ELECTION [ input=I_ELECTION cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_INTEGRATE_TIMER_STOP
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_FINALIZE_TIMER_STOP
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_VOTE
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: crm_uptime: Current CPU usage is: 0s, 10000us
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_election_vote: Started election 2
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: crm_timer_start: Started Election Timeout (I_ELECTION_DC:120000ms), src=16
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_COUNT
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_election_count_vote: Created voted hash
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: crm_uptime: Current CPU usage is: 0s, 10000us
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_election_count_vote: Election 2 (current: 2, owner: Cluster-Server-1): Processed vote from Cluster-Server-1 (Recorded)
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_CHECK
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_election_check: Still waiting on 1 non-votes (2 total)
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_COUNT
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: crm_uptime: Current CPU usage is: 0s, 10000us
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: crm_compare_age: Loose: 10000 vs 20000 (usec)
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: info: do_election_count_vote: Election 3 (owner: Cluster-Server-2) lost: vote from Cluster-Server-2 (Uptime)
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_CHECK
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_election_check: Still waiting on 1 non-votes (2 total)
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_PENDING: [ state=S_ELECTION cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: notice: do_state_transition: State transition S_ELECTION -> S_PENDING [ input=I_PENDING cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_INTEGRATE_TIMER_STOP
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_FINALIZE_TIMER_STOP
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_START
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: crm_timer_start: Started Election Trigger (I_DC_TIMEOUT:20000ms), src=17
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_RELEASE
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_dc_release: Releasing the role of DC
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_RELEASED
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: info: do_dc_release: DC role released
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_PE_STOP
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_TE_STOP
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: cib_client_del_notify_callback: Removing callback for cib_diff_notify events
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: info: do_te_control: Transitioner is now inactive
Sep 10 15:21:40 Cluster-Server-1 cib: [48709]: debug: cib_common_callback_worker: Setting cib_diff_notify callbacks for 48715 (e00c5ff0-ca9f-4fba-a6df-4b13f36ff633): off
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_RELEASE_SUCCESS: [ state=S_PENDING cause=C_FSA_INTERNAL origin=do_dc_release ]
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_LOG   
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_log: FSA: Input I_RELEASE_SUCCESS from do_dc_release() received in state S_PENDING
Sep 10 15:21:40 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for cib_apply_diff op
Sep 10 15:21:40 Cluster-Server-1 cib: [48709]: debug: Forking temp process write_cib_contents
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: handle_request: Raising I_JOIN_OFFER: join-1
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_JOIN_OFFER: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ]
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_CL_JOIN_REQUEST
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: info: update_dc: Set DC to Cluster-Server-2 (3.0.6)
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_offer_respond: do_cl_join_offer_respond added action A_DC_TIMER_STOP to the FSA
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:21:40 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for cib_apply_diff op
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: join_query_callback: Respond to join offer join-1
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: join_query_callback: Acknowledging Cluster-Server-2 as our DC
Sep 10 15:21:40 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for cib_replace op
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: handle_request: Raising I_JOIN_RESULT: join-1
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_JOIN_RESULT: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ]
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_CL_JOIN_RESULT
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_finalize_respond: Confirming join join-1: join_ack_nack
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_finalize_respond: join-1: Join complete.  Sending local LRM status to Cluster-Server-2
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: info: erase_status_tag: Deleting xpath: //node_state[@uname='Cluster-Server-1']/transient_attributes
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: info: update_attrd: Connecting to attrd...
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:21:40 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for cib_apply_diff op
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: attrd_update_delegate: Sent update: terminate=(null) for Cluster-Server-1
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: attrd_update_delegate: Sent update: shutdown=(null) for Cluster-Server-1
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: attrd_update_delegate: Sent update: (null)=(null) for localhost
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_NOT_DC: [ state=S_PENDING cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]
Sep 10 15:21:40 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crmd: terminate=<null>
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_LOG   
Sep 10 15:21:40 Cluster-Server-1 attrd: [48713]: info: find_hash_entry: Creating hash entry for terminate
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_log: FSA: Input I_NOT_DC from do_cl_join_finalize_respond() received in state S_PENDING
Sep 10 15:21:40 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: (null), Current: (null), Stored: (null)
Sep 10 15:21:40 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crmd: shutdown=<null>
Sep 10 15:21:40 Cluster-Server-1 attrd: [48713]: info: find_hash_entry: Creating hash entry for shutdown
Sep 10 15:21:40 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: (null), Current: (null), Stored: (null)
Sep 10 15:21:40 Cluster-Server-1 attrd: [48713]: notice: attrd_local_callback: Sending full refresh (origin=crmd)
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: notice: do_state_transition: State transition S_PENDING -> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_INTEGRATE_TIMER_STOP
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_FINALIZE_TIMER_STOP
Sep 10 15:21:40 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for cib_apply_diff op
Sep 10 15:21:40 Cluster-Server-1 cib: [48709]: info: Managed write_cib_contents process 49222 exited with return code 0.
Sep 10 15:21:40 Cluster-Server-1 cib: [48709]: debug: Forking temp process write_cib_contents
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: erase_xpath_callback: Deletion of "//node_state[@uname='Cluster-Server-1']/transient_attributes": ok (rc=0)
Sep 10 15:21:40 Cluster-Server-1 crmd: [48715]: debug: attrd_update_delegate: Sent update: probe_complete=true for localhost
Sep 10 15:21:40 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crmd: probe_complete=true
Sep 10 15:21:40 Cluster-Server-1 attrd: [48713]: info: find_hash_entry: Creating hash entry for probe_complete
Sep 10 15:21:40 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: true, Current: (null), Stored: (null)
Sep 10 15:21:40 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: New value of probe_complete is true
Sep 10 15:21:40 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
Sep 10 15:21:40 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] does not exist
Sep 10 15:21:40 Cluster-Server-1 cib: [48709]: info: Managed write_cib_contents process 49223 exited with return code 0.
Sep 10 15:21:40 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for /cib (/cib)
Sep 10 15:21:40 Cluster-Server-1 attrd: [48713]: notice: attrd_perform_update: Sent update 4: probe_complete=true
Sep 10 15:21:40 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] does not exist
Sep 10 15:21:40 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for /cib (/cib)
Sep 10 15:21:40 Cluster-Server-1 attrd: [48713]: notice: attrd_perform_update: Sent update 7: probe_complete=true
Sep 10 15:21:40 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 4 for probe_complete=true passed
Sep 10 15:21:40 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 7 for probe_complete=true passed
Sep 10 15:21:41 Cluster-Server-1 cib: [48709]: info: apply_xml_diff: Digest mis-match: expected bdeb17841a8a81071a45941b740ffab0, calculated 7cefe735f2b561e5c0292c1589ee7e8c
Sep 10 15:21:41 Cluster-Server-1 cib: [48709]: notice: cib_process_diff: Diff 0.4.14 -> 0.5.1 not applied to 0.4.14: Failed application of an update diff
Sep 10 15:21:41 Cluster-Server-1 cib: [48709]: info: cib_server_process_diff: Requesting re-sync from peer
Sep 10 15:21:41 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair)
Sep 10 15:21:41 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_COUNT
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: crm_uptime: Current CPU usage is: 0s, 10000us
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: crm_compare_age: Loose: 10000 vs 30000 (usec)
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: info: do_election_count_vote: Election 4 (owner: Cluster-Server-2) lost: vote from Cluster-Server-2 (Uptime)
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: update_dc: Unset DC. Was Cluster-Server-2
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_CHECK
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: do_election_check: Ignore election check: we not in an election
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_PENDING: [ state=S_NOT_DC cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_LOG   
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: do_log: FSA: Input I_PENDING from do_election_count_vote() received in state S_NOT_DC
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: notice: do_state_transition: State transition S_NOT_DC -> S_PENDING [ input=I_PENDING cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_INTEGRATE_TIMER_STOP
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_FINALIZE_TIMER_STOP
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_START
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: crm_timer_start: Started Election Trigger (I_DC_TIMEOUT:20000ms), src=21
Sep 10 15:21:41 Cluster-Server-1 cib: [48709]: notice: cib_server_process_diff: Not applying diff 0.5.1 -> 0.5.2 (sync in progress)
Sep 10 15:21:41 Cluster-Server-1 cib: [48709]: notice: cib_server_process_diff: Not applying diff 0.5.2 -> 0.5.3 (sync in progress)
Sep 10 15:21:41 Cluster-Server-1 cib: [48709]: notice: cib_server_process_diff: Not applying diff 0.5.3 -> 0.5.4 (sync in progress)
Sep 10 15:21:41 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for cib_replace op
Sep 10 15:21:41 Cluster-Server-1 cib: [48709]: info: cib_replace_notify: Replaced: -1.-1.-1 -> 0.5.4 from Cluster-Server-2
Sep 10 15:21:41 Cluster-Server-1 attrd: [48713]: info: do_cib_replaced: Sending full refresh
Sep 10 15:21:41 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: do_cib_replaced: Updating the CIB after a replace: DC=false
Sep 10 15:21:41 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair)
Sep 10 15:21:41 Cluster-Server-1 cib: [48709]: debug: Forking temp process write_cib_contents
Sep 10 15:21:41 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:21:41 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 9 for probe_complete=true passed
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: handle_request: Raising I_JOIN_OFFER: join-2
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_JOIN_OFFER: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ]
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_CL_JOIN_REQUEST
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: info: update_dc: Set DC to Cluster-Server-2 (3.0.6)
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_offer_respond: do_cl_join_offer_respond added action A_DC_TIMER_STOP to the FSA
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:21:41 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 11 for probe_complete=true passed
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: join_query_callback: Respond to join offer join-2
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: join_query_callback: Acknowledging Cluster-Server-2 as our DC
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: handle_request: Raising I_JOIN_RESULT: join-2
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_JOIN_RESULT: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ]
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_CL_JOIN_RESULT
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_finalize_respond: Confirming join join-2: join_ack_nack
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_finalize_respond: join-2: Join complete.  Sending local LRM status to Cluster-Server-2
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: attrd_update_delegate: Sent update: (null)=(null) for localhost
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_NOT_DC: [ state=S_PENDING cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_LOG   
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: do_log: FSA: Input I_NOT_DC from do_cl_join_finalize_respond() received in state S_PENDING
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: notice: do_state_transition: State transition S_PENDING -> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_INTEGRATE_TIMER_STOP
Sep 10 15:21:41 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_FINALIZE_TIMER_STOP
Sep 10 15:21:41 Cluster-Server-1 attrd: [48713]: notice: attrd_local_callback: Sending full refresh (origin=crmd)
Sep 10 15:21:41 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
Sep 10 15:21:41 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for cib_replace op
Sep 10 15:21:42 Cluster-Server-1 cib: [48709]: info: Managed write_cib_contents process 49299 exited with return code 0.
Sep 10 15:21:42 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair)
Sep 10 15:21:42 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:21:42 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair)
Sep 10 15:21:42 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:21:42 Cluster-Server-1 cib: [48709]: debug: Forking temp process write_cib_contents
Sep 10 15:21:42 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 13 for probe_complete=true passed
Sep 10 15:21:42 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 15 for probe_complete=true passed
Sep 10 15:21:42 Cluster-Server-1 cib: [48709]: info: Managed write_cib_contents process 49302 exited with return code 0.
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [49336] registered
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:49336] disconnected.
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:49336] is unregistered
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [49338] registered
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:49338] disconnected.
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:49338] is unregistered
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [49340] registered
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:49340] disconnected.
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:49340] is unregistered
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [49342] registered
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:49342] disconnected.
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:49342] is unregistered
Sep 10 15:21:43 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for cib_apply_diff op
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_COUNT
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: crm_uptime: Current CPU usage is: 0s, 10000us
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: crm_compare_age: Loose: 10000 vs 50000 (usec)
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: info: do_election_count_vote: Election 5 (owner: Cluster-Server-2) lost: vote from Cluster-Server-2 (Uptime)
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: update_dc: Unset DC. Was Cluster-Server-2
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_CHECK
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: do_election_check: Ignore election check: we not in an election
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_PENDING: [ state=S_NOT_DC cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_LOG   
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: do_log: FSA: Input I_PENDING from do_election_count_vote() received in state S_NOT_DC
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: notice: do_state_transition: State transition S_NOT_DC -> S_PENDING [ input=I_PENDING cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_INTEGRATE_TIMER_STOP
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_FINALIZE_TIMER_STOP
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_START
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: crm_timer_start: Started Election Trigger (I_DC_TIMEOUT:20000ms), src=23
Sep 10 15:21:43 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair)
Sep 10 15:21:43 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:21:43 Cluster-Server-1 cib: [48709]: debug: Forking temp process write_cib_contents
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: handle_request: Raising I_JOIN_OFFER: join-3
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_JOIN_OFFER: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ]
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_CL_JOIN_REQUEST
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: info: update_dc: Set DC to Cluster-Server-2 (3.0.6)
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_offer_respond: do_cl_join_offer_respond added action A_DC_TIMER_STOP to the FSA
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:21:43 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 17 for probe_complete=true passed
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: join_query_callback: Respond to join offer join-3
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: join_query_callback: Acknowledging Cluster-Server-2 as our DC
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: handle_request: Raising I_JOIN_RESULT: join-3
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_JOIN_RESULT: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ]
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_CL_JOIN_RESULT
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_finalize_respond: Confirming join join-3: join_ack_nack
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_finalize_respond: join-3: Join complete.  Sending local LRM status to Cluster-Server-2
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: attrd_update_delegate: Sent update: (null)=(null) for localhost
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_NOT_DC: [ state=S_PENDING cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_LOG   
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: do_log: FSA: Input I_NOT_DC from do_cl_join_finalize_respond() received in state S_PENDING
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: notice: do_state_transition: State transition S_PENDING -> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:21:43 Cluster-Server-1 attrd: [48713]: notice: attrd_local_callback: Sending full refresh (origin=crmd)
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_INTEGRATE_TIMER_STOP
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_FINALIZE_TIMER_STOP
Sep 10 15:21:43 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
Sep 10 15:21:43 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for cib_replace op
Sep 10 15:21:43 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair)
Sep 10 15:21:43 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:21:43 Cluster-Server-1 cib: [48709]: info: Managed write_cib_contents process 49356 exited with return code 0.
Sep 10 15:21:43 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 19 for probe_complete=true passed
Sep 10 15:21:43 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair)
Sep 10 15:21:43 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:21:43 Cluster-Server-1 cib: [48709]: debug: Forking temp process write_cib_contents
Sep 10 15:21:43 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 21 for probe_complete=true passed
Sep 10 15:21:43 Cluster-Server-1 cib: [48709]: info: Managed write_cib_contents process 49359 exited with return code 0.
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: debug: on_msg_add_rsc:client [48715] adds resource p_NFS_Server:0
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: notice: lrmd_rsc_new(): No lrm_rprovider field in message
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=4:2:7:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=p_NFS_Server:0_monitor_0
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc p_NFS_Server:0
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[2] on p_NFS_Server:0 for client 48715, its parameters: crm_feature_set=[3.0.6] CRM_meta_notify=[false] CRM_meta_clone=[0] CRM_meta_globally_unique=[false] CRM_meta_clone_node_max=[1] CRM_meta_timeout=[20000] CRM_meta_clone_max=[2]  to the operation list.
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: info: rsc:p_NFS_Server:0 probe[2] (pid 49362)
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: debug: on_msg_add_rsc:client [48715] adds resource p_iSCSI_Daemon:0
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: notice: lrmd_rsc_new(): No lrm_rprovider field in message
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=5:2:7:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=p_iSCSI_Daemon:0_monitor_0
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc p_iSCSI_Daemon:0
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[3] on p_iSCSI_Daemon:0 for client 48715, its parameters: crm_feature_set=[3.0.6] CRM_meta_notify=[false] CRM_meta_clone=[0] CRM_meta_globally_unique=[false] CRM_meta_clone_node_max=[1] CRM_meta_timeout=[20000] CRM_meta_clone_max=[2]  to the operation list.
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: info: rsc:p_iSCSI_Daemon:0 probe[3] (pid 49363)
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: debug: on_msg_add_rsc:client [48715] adds resource p_PingD:0
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=6:2:7:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=p_PingD:0_monitor_0
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc p_PingD:0
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[4] on p_PingD:0 for client 48715, its parameters: CRM_meta_timeout=[20000] multiplier=[100] CRM_meta_clone_node_max=[1] CRM_meta_notify=[false] crm_feature_set=[3.0.6] CRM_meta_clone=[0] host_list=[192.168.1.1] CRM_meta_clone_max=[2] CRM_meta_globally_unique=[false]  to the operation list.
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: info: rsc:p_PingD:0 probe[4] (pid 49364)
Sep 10 15:21:43 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for cib_replace op
Sep 10 15:21:43 Cluster-Server-1 cib: [48709]: WARN: send_ipc_message: IPC Channel to 49361 is not connected
Sep 10 15:21:43 Cluster-Server-1 cib: [48709]: WARN: send_via_callback_channel: Delivery of reply to client cibadmin/7966687b-6794-4753-b783-bd4a31539744 failed
Sep 10 15:21:43 Cluster-Server-1 cib: [48709]: WARN: do_local_notify: Sync reply to cibadmin failed: reply failed
Sep 10 15:21:43 Cluster-Server-1 cib: [48709]: debug: Forking temp process write_cib_contents
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: info: Managed p_iSCSI_Daemon:0:monitor process 49363 exited with return code 0.
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: info: operation monitor[3] on p_iSCSI_Daemon:0 for client 48715: pid 49363 exited with return code 0
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce p_iSCSI_Daemon:0 after complete monitor op (interval=0)
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation p_iSCSI_Daemon:0_monitor_0 (call=3, rc=0, cib-update=15, confirmed=true) ok
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'p_iSCSI_Daemon:0'
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: WARN: Managed p_NFS_Server:0:monitor process 49362 exited with return code 3.
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: info: operation monitor[2] on p_NFS_Server:0 for client 48715: pid 49362 exited with return code 7 (mapped from 3)
Sep 10 15:21:43 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_NFS_Server:0:monitor:stdout) nfsd not running

Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce p_NFS_Server:0 after complete monitor op (interval=0)
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation p_NFS_Server:0_monitor_0 (call=2, rc=7, cib-update=16, confirmed=true) not running
Sep 10 15:21:43 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'p_NFS_Server:0'
Sep 10 15:21:43 Cluster-Server-1 cib: [48709]: info: Managed write_cib_contents process 49371 exited with return code 0.
Sep 10 15:21:44 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for cib_replace op
Sep 10 15:21:44 Cluster-Server-1 cib: [48709]: debug: Forking temp process write_cib_contents
Sep 10 15:21:44 Cluster-Server-1 cib: [48709]: info: Managed write_cib_contents process 49392 exited with return code 0.
Sep 10 15:21:45 Cluster-Server-1 attrd_updater: [49395]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:21:45 Cluster-Server-1 attrd_updater: [49395]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:21:45 Cluster-Server-1 attrd_updater: [49395]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:21:45 Cluster-Server-1 attrd_updater: [49395]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:21:45 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:21:45 Cluster-Server-1 attrd: [48713]: info: find_hash_entry: Creating hash entry for pingd
Sep 10 15:21:45 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: (null), Stored: (null)
Sep 10 15:21:45 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: New value of pingd is 100
Sep 10 15:21:45 Cluster-Server-1 lrmd: [48712]: info: Managed p_PingD:0:monitor process 49364 exited with return code 0.
Sep 10 15:21:45 Cluster-Server-1 lrmd: [48712]: info: operation monitor[4] on p_PingD:0 for client 48715: pid 49364 exited with return code 0
Sep 10 15:21:45 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce p_PingD:0 after complete monitor op (interval=0)
Sep 10 15:21:45 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation p_PingD:0_monitor_0 (call=4, rc=0, cib-update=17, confirmed=true) ok
Sep 10 15:21:45 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'p_PingD:0'
Sep 10 15:21:45 Cluster-Server-1 crmd: [48715]: debug: attrd_update_delegate: Sent update: probe_complete=true for localhost
Sep 10 15:21:45 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crmd: probe_complete=true
Sep 10 15:21:45 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: true, Current: true, Stored: true
Sep 10 15:21:45 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=15:3:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=p_iSCSI_Daemon:0_monitor_30000
Sep 10 15:21:45 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[5] on p_iSCSI_Daemon:0 for client 48715, its parameters: CRM_meta_timeout=[20000] CRM_meta_name=[monitor] crm_feature_set=[3.0.6] CRM_meta_notify=[false] CRM_meta_clone_node_max=[1] CRM_meta_clone=[0] CRM_meta_clone_max=[2] CRM_meta_interval=[30000] CRM_meta_globally_unique=[false]  to the operation list.
Sep 10 15:21:45 Cluster-Server-1 lrmd: [48712]: info: rsc:p_iSCSI_Daemon:0 monitor[5] (pid 49403)
Sep 10 15:21:45 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=25:3:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=p_PingD:0_monitor_10000
Sep 10 15:21:45 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[6] on p_PingD:0 for client 48715, its parameters: CRM_meta_timeout=[5000] multiplier=[100] CRM_meta_name=[monitor] CRM_meta_clone_node_max=[1] CRM_meta_notify=[false] crm_feature_set=[3.0.6] CRM_meta_clone=[0] host_list=[192.168.1.1] CRM_meta_clone_max=[2] CRM_meta_interval=[10000] CRM_meta_globally_unique=[false]  to the operation list.
Sep 10 15:21:45 Cluster-Server-1 lrmd: [48712]: info: rsc:p_PingD:0 monitor[6] (pid 49404)
Sep 10 15:21:45 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=5:3:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=p_NFS_Server:0_start_0
Sep 10 15:21:45 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc p_NFS_Server:0
Sep 10 15:21:45 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation start[7] on p_NFS_Server:0 for client 48715, its parameters: crm_feature_set=[3.0.6] CRM_meta_notify=[false] CRM_meta_clone=[0] CRM_meta_globally_unique=[false] CRM_meta_clone_node_max=[1] CRM_meta_clone_max=[2] CRM_meta_timeout=[20000]  to the operation list.
Sep 10 15:21:45 Cluster-Server-1 lrmd: [48712]: info: rsc:p_NFS_Server:0 start[7] (pid 49408)
Sep 10 15:21:45 Cluster-Server-1 lrmd: [49408]: WARN: For LSB init script, no additional parameters are needed.
Sep 10 15:21:45 Cluster-Server-1 lrmd: [48712]: info: Managed p_iSCSI_Daemon:0:monitor process 49403 exited with return code 0.
Sep 10 15:21:45 Cluster-Server-1 lrmd: [48712]: info: operation monitor[5] on p_iSCSI_Daemon:0 for client 48715: pid 49403 exited with return code 0
Sep 10 15:21:45 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce p_iSCSI_Daemon:0 after complete monitor op (interval=30000)
Sep 10 15:21:45 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation p_iSCSI_Daemon:0_monitor_30000 (call=5, rc=0, cib-update=18, confirmed=false) ok
Sep 10 15:21:45 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'p_iSCSI_Daemon:0'
Sep 10 15:21:45 Cluster-Server-1 lrmd: [48712]: info: RA output: (p_NFS_Server:0:start:stdout) Exporting directories for NFS kernel daemon...
Sep 10 15:21:45 Cluster-Server-1 lrmd: [48712]: info: RA output: (p_NFS_Server:0:start:stdout) .

Sep 10 15:21:45 Cluster-Server-1 lrmd: [48712]: info: RA output: (p_NFS_Server:0:start:stdout) Starting NFS kernel daemon:
Sep 10 15:21:45 Cluster-Server-1 lrmd: [48712]: info: RA output: (p_NFS_Server:0:start:stdout)  nfsd
Sep 10 15:21:45 Cluster-Server-1 lrmd: [48712]: info: RA output: (p_NFS_Server:0:start:stdout)  mountd
Sep 10 15:21:45 Cluster-Server-1 lrmd: [48712]: info: RA output: (p_NFS_Server:0:start:stdout) .

Sep 10 15:21:45 Cluster-Server-1 lrmd: [48712]: info: Managed p_NFS_Server:0:start process 49408 exited with return code 0.
Sep 10 15:21:45 Cluster-Server-1 lrmd: [48712]: info: operation start[7] on p_NFS_Server:0 for client 48715: pid 49408 exited with return code 0
Sep 10 15:21:45 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce p_NFS_Server:0 after complete start op (interval=0)
Sep 10 15:21:45 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation p_NFS_Server:0_start_0 (call=7, rc=0, cib-update=19, confirmed=true) ok
Sep 10 15:21:45 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending start op to history for 'p_NFS_Server:0'
Sep 10 15:21:45 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=6:3:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=p_NFS_Server:0_monitor_30000
Sep 10 15:21:45 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[8] on p_NFS_Server:0 for client 48715, its parameters: CRM_meta_timeout=[20000] CRM_meta_name=[monitor] crm_feature_set=[3.0.6] CRM_meta_notify=[false] CRM_meta_clone_node_max=[1] CRM_meta_clone=[0] CRM_meta_clone_max=[2] CRM_meta_interval=[30000] CRM_meta_globally_unique=[false]  to the operation list.
Sep 10 15:21:45 Cluster-Server-1 lrmd: [48712]: info: rsc:p_NFS_Server:0 monitor[8] (pid 49448)
Sep 10 15:21:45 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_NFS_Server:0:monitor:stdout) nfsd running

Sep 10 15:21:45 Cluster-Server-1 lrmd: [48712]: info: Managed p_NFS_Server:0:monitor process 49448 exited with return code 0.
Sep 10 15:21:45 Cluster-Server-1 lrmd: [48712]: info: operation monitor[8] on p_NFS_Server:0 for client 48715: pid 49448 exited with return code 0
Sep 10 15:21:45 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce p_NFS_Server:0 after complete monitor op (interval=30000)
Sep 10 15:21:45 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation p_NFS_Server:0_monitor_30000 (call=8, rc=0, cib-update=20, confirmed=false) ok
Sep 10 15:21:45 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'p_NFS_Server:0'
Sep 10 15:21:47 Cluster-Server-1 attrd_updater: [49509]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:21:47 Cluster-Server-1 attrd_updater: [49509]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:21:47 Cluster-Server-1 attrd_updater: [49509]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:21:47 Cluster-Server-1 attrd_updater: [49509]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:21:47 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:21:47 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: (null)
Sep 10 15:21:47 Cluster-Server-1 lrmd: [48712]: info: Managed p_PingD:0:monitor process 49404 exited with return code 0.
Sep 10 15:21:47 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 49404 exited with return code 0
Sep 10 15:21:47 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce p_PingD:0 after complete monitor op (interval=10000)
Sep 10 15:21:47 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation p_PingD:0_monitor_10000 (call=6, rc=0, cib-update=21, confirmed=false) ok
Sep 10 15:21:47 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'p_PingD:0'
Sep 10 15:21:50 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] does not exist
Sep 10 15:21:50 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for /cib (/cib)
Sep 10 15:21:50 Cluster-Server-1 attrd: [48713]: notice: attrd_perform_update: Sent update 24: pingd=100
Sep 10 15:21:50 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 24 for pingd=100 passed
Sep 10 15:21:57 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 50535)
Sep 10 15:21:59 Cluster-Server-1 attrd_updater: [50553]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:21:59 Cluster-Server-1 attrd_updater: [50553]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:21:59 Cluster-Server-1 attrd_updater: [50553]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:21:59 Cluster-Server-1 attrd_updater: [50553]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:21:59 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:21:59 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:21:59 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 50535 exited with return code 0
Sep 10 15:22:06 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [50894] registered
Sep 10 15:22:06 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:50894] disconnected.
Sep 10 15:22:06 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:50894] is unregistered
Sep 10 15:22:06 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [50896] registered
Sep 10 15:22:06 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:50896] disconnected.
Sep 10 15:22:06 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:50896] is unregistered
Sep 10 15:22:06 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [50898] registered
Sep 10 15:22:06 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:50898] disconnected.
Sep 10 15:22:06 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:50898] is unregistered
Sep 10 15:22:06 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [50900] registered
Sep 10 15:22:06 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:50900] disconnected.
Sep 10 15:22:06 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:50900] is unregistered
Sep 10 15:22:06 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [50909] registered
Sep 10 15:22:06 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:50909] disconnected.
Sep 10 15:22:06 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:50909] is unregistered
Sep 10 15:22:06 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [50918] registered
Sep 10 15:22:06 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:50918] disconnected.
Sep 10 15:22:06 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:50918] is unregistered
Sep 10 15:22:06 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_COUNT
Sep 10 15:22:06 Cluster-Server-1 crmd: [48715]: debug: crm_uptime: Current CPU usage is: 0s, 20000us
Sep 10 15:22:06 Cluster-Server-1 crmd: [48715]: debug: crm_compare_age: Loose: 20000 vs 100000 (usec)
Sep 10 15:22:06 Cluster-Server-1 crmd: [48715]: info: do_election_count_vote: Election 6 (owner: Cluster-Server-2) lost: vote from Cluster-Server-2 (Uptime)
Sep 10 15:22:06 Cluster-Server-1 crmd: [48715]: debug: update_dc: Unset DC. Was Cluster-Server-2
Sep 10 15:22:06 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_CHECK
Sep 10 15:22:06 Cluster-Server-1 crmd: [48715]: debug: do_election_check: Ignore election check: we not in an election
Sep 10 15:22:06 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_PENDING: [ state=S_NOT_DC cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Sep 10 15:22:06 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_LOG   
Sep 10 15:22:06 Cluster-Server-1 crmd: [48715]: debug: do_log: FSA: Input I_PENDING from do_election_count_vote() received in state S_NOT_DC
Sep 10 15:22:06 Cluster-Server-1 crmd: [48715]: notice: do_state_transition: State transition S_NOT_DC -> S_PENDING [ input=I_PENDING cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Sep 10 15:22:06 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:22:06 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_INTEGRATE_TIMER_STOP
Sep 10 15:22:06 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_FINALIZE_TIMER_STOP
Sep 10 15:22:06 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_START
Sep 10 15:22:06 Cluster-Server-1 crmd: [48715]: debug: crm_timer_start: Started Election Trigger (I_DC_TIMEOUT:20000ms), src=32
Sep 10 15:22:06 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for cib_apply_diff op
Sep 10 15:22:06 Cluster-Server-1 cib: [48709]: debug: Forking temp process write_cib_contents
Sep 10 15:22:06 Cluster-Server-1 crmd: [48715]: debug: handle_request: Raising I_JOIN_OFFER: join-4
Sep 10 15:22:06 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_JOIN_OFFER: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ]
Sep 10 15:22:06 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:22:06 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_CL_JOIN_REQUEST
Sep 10 15:22:06 Cluster-Server-1 crmd: [48715]: info: update_dc: Set DC to Cluster-Server-2 (3.0.6)
Sep 10 15:22:06 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_offer_respond: do_cl_join_offer_respond added action A_DC_TIMER_STOP to the FSA
Sep 10 15:22:06 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:22:07 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:22:07 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:22:07 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:22:07 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: join_query_callback: Respond to join offer join-4
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: join_query_callback: Acknowledging Cluster-Server-2 as our DC
Sep 10 15:22:07 Cluster-Server-1 cib: [48709]: info: Managed write_cib_contents process 50931 exited with return code 0.
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: handle_request: Raising I_JOIN_RESULT: join-4
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_JOIN_RESULT: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ]
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_CL_JOIN_RESULT
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_finalize_respond: Confirming join join-4: join_ack_nack
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_NFS_Server:0 after complete start op (interval=0)
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_NFS_Server:0 after complete monitor op (interval=30000)
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_PingD:0 after complete monitor op (interval=0)
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_PingD:0 after complete monitor op (interval=10000)
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_iSCSI_Daemon:0 after complete monitor op (interval=0)
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_iSCSI_Daemon:0 after complete monitor op (interval=30000)
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_finalize_respond: join-4: Join complete.  Sending local LRM status to Cluster-Server-2
Sep 10 15:22:07 Cluster-Server-1 attrd: [48713]: notice: attrd_local_callback: Sending full refresh (origin=crmd)
Sep 10 15:22:07 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: attrd_update_delegate: Sent update: (null)=(null) for localhost
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_NOT_DC: [ state=S_PENDING cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_LOG   
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: do_log: FSA: Input I_NOT_DC from do_cl_join_finalize_respond() received in state S_PENDING
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: notice: do_state_transition: State transition S_PENDING -> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_INTEGRATE_TIMER_STOP
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_FINALIZE_TIMER_STOP
Sep 10 15:22:07 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for cib_replace op
Sep 10 15:22:07 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:22:07 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:22:07 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: pingd (100)
Sep 10 15:22:07 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 26 for probe_complete=true passed
Sep 10 15:22:07 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 28 for pingd=100 passed
Sep 10 15:22:07 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:22:07 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:22:07 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:22:07 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:22:07 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:22:07 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:22:07 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 30 for probe_complete=true passed
Sep 10 15:22:07 Cluster-Server-1 cib: [48709]: debug: Forking temp process write_cib_contents
Sep 10 15:22:07 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 32 for pingd=100 passed
Sep 10 15:22:07 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 34 for probe_complete=true passed
Sep 10 15:22:07 Cluster-Server-1 lrmd: [48712]: debug: on_msg_add_rsc:client [48715] adds resource p_Device_drive:0
Sep 10 15:22:07 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 36 for pingd=100 passed
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=10:5:7:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=p_Device_drive:0_monitor_0
Sep 10 15:22:07 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc p_Device_drive:0
Sep 10 15:22:07 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[9] on p_Device_drive:0 for client 48715, its parameters: drbd_resource=[drive] CRM_meta_timeout=[20000] CRM_meta_clone_node_max=[1] CRM_meta_notify=[true] crm_feature_set=[3.0.6] CRM_meta_clone=[0] CRM_meta_clone_max=[2] CRM_meta_master_node_max=[1] CRM_meta_globally_unique=[false] CRM_meta_master_max=[1]  to the operation list.
Sep 10 15:22:07 Cluster-Server-1 lrmd: [48712]: info: rsc:p_Device_drive:0 probe[9] (pid 50936)
Sep 10 15:22:07 Cluster-Server-1 lrmd: [48712]: debug: on_msg_add_rsc:client [48715] adds resource LVM_drive
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=11:5:7:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=LVM_drive_monitor_0
Sep 10 15:22:07 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc LVM_drive
Sep 10 15:22:07 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[10] on LVM_drive for client 48715, its parameters: crm_feature_set=[3.0.6] activation_mode=[auto] vg_name=[drive-CSD] CRM_meta_timeout=[20000]  to the operation list.
Sep 10 15:22:07 Cluster-Server-1 lrmd: [48712]: info: rsc:LVM_drive probe[10] (pid 50939)
Sep 10 15:22:07 Cluster-Server-1 cib: [48709]: info: Managed write_cib_contents process 50934 exited with return code 0.
drbd(p_Device_drive:0)[50936]:	2012/09/10_15:22:07 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:22:07 Cluster-Server-1 lrmd: [48712]: info: Managed LVM_drive:monitor process 50939 exited with return code 0.
Sep 10 15:22:07 Cluster-Server-1 lrmd: [48712]: info: operation monitor[10] on LVM_drive for client 48715: pid 50939 exited with return code 0
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce LVM_drive after complete monitor op (interval=0)
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51000]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51000]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51000]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51000]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51000]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51000]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51000]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51000]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51000]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51000]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:22:07 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:22:07 Cluster-Server-1 attrd: [48713]: info: find_hash_entry: Creating hash entry for master-p_Device_drive:0
Sep 10 15:22:07 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: (null), Stored: (null)
Sep 10 15:22:07 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: New value of master-p_Device_drive:0 is 10000
Sep 10 15:22:07 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: master-p_Device_drive:0 (10000)
Sep 10 15:22:07 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:0'] does not exist
Sep 10 15:22:07 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for /cib (/cib)
Sep 10 15:22:07 Cluster-Server-1 attrd: [48713]: notice: attrd_perform_update: Sent update 39: master-p_Device_drive:0=10000
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51000]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51000]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51000]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51000]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51000]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:22:07 Cluster-Server-1 attrd: [48713]: info: find_hash_entry: Creating hash entry for master-p_Device_drive:1
Sep 10 15:22:07 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:1'] does not exist
Sep 10 15:22:07 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update -22 for master-p_Device_drive:1=(null) passed
drbd(p_Device_drive:0)[50936]:	2012/09/10_15:22:07 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[50936]:	2012/09/10_15:22:07 DEBUG: drive: Command output: 
Sep 10 15:22:07 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 39 for master-p_Device_drive:0=10000 passed
Sep 10 15:22:07 Cluster-Server-1 lrmd: [48712]: WARN: Managed p_Device_drive:0:monitor process 50936 exited with return code 8.
Sep 10 15:22:07 Cluster-Server-1 lrmd: [48712]: info: operation monitor[9] on p_Device_drive:0 for client 48715: pid 50936 exited with return code 8
Sep 10 15:22:07 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation LVM_drive_monitor_0 (call=10, rc=0, cib-update=25, confirmed=true) ok
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'LVM_drive'
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce p_Device_drive:0 after complete monitor op (interval=0)
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation p_Device_drive:0_monitor_0 (call=9, rc=8, cib-update=26, confirmed=true) master
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'p_Device_drive:0'
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: attrd_update_delegate: Sent update: probe_complete=true for localhost
Sep 10 15:22:07 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crmd: probe_complete=true
Sep 10 15:22:07 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: true, Current: true, Stored: true
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=39:6:8:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=p_Device_drive:0_monitor_10000
Sep 10 15:22:07 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[11] on p_Device_drive:0 for client 48715, its parameters: drbd_resource=[drive] CRM_meta_role=[Master] CRM_meta_timeout=[20000] CRM_meta_name=[monitor] CRM_meta_clone_node_max=[1] CRM_meta_notify=[true] crm_feature_set=[3.0.6] CRM_meta_clone=[0] CRM_meta_clone_max=[2] CRM_meta_master_node_max=[1] CRM_meta_interval=[10000] CRM_meta_globally_unique=[false] CRM_meta_master_max=[1]  to the operation list.
Sep 10 15:22:07 Cluster-Server-1 lrmd: [48712]: info: rsc:p_Device_drive:0 monitor[11] (pid 51019)
drbd(p_Device_drive:0)[51019]:	2012/09/10_15:22:07 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51049]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51049]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51049]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51049]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51049]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51049]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51049]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51049]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51049]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51049]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51049]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51049]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51049]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51049]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:22:07 Cluster-Server-1 crm_attribute: [51049]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:22:07 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:22:07 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[51019]:	2012/09/10_15:22:07 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[51019]:	2012/09/10_15:22:07 DEBUG: drive: Command output: 
Sep 10 15:22:07 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:22:07 Cluster-Server-1 lrmd: [48712]: WARN: Managed p_Device_drive:0:monitor process 51019 exited with return code 8.
Sep 10 15:22:07 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 51019 exited with return code 8
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce p_Device_drive:0 after complete monitor op (interval=10000)
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation p_Device_drive:0_monitor_10000 (call=11, rc=8, cib-update=27, confirmed=false) master
Sep 10 15:22:07 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'p_Device_drive:0'
Sep 10 15:22:09 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 51081)
Sep 10 15:22:11 Cluster-Server-1 attrd_updater: [51104]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:22:11 Cluster-Server-1 attrd_updater: [51104]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:22:11 Cluster-Server-1 attrd_updater: [51104]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:22:11 Cluster-Server-1 attrd_updater: [51104]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:22:11 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:22:11 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:22:11 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 51081 exited with return code 0
Sep 10 15:22:15 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_iSCSI_Daemon:0 monitor[5] (pid 51105)
Sep 10 15:22:15 Cluster-Server-1 lrmd: [48712]: info: operation monitor[5] on p_iSCSI_Daemon:0 for client 48715: pid 51105 exited with return code 0
Sep 10 15:22:15 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_NFS_Server:0 monitor[8] (pid 51109)
Sep 10 15:22:15 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_NFS_Server:0:monitor:stdout) nfsd running

Sep 10 15:22:15 Cluster-Server-1 lrmd: [48712]: info: operation monitor[8] on p_NFS_Server:0 for client 48715: pid 51109 exited with return code 0
Sep 10 15:22:17 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 51111)
drbd(p_Device_drive:0)[51111]:	2012/09/10_15:22:17 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:22:17 Cluster-Server-1 crm_attribute: [51141]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:22:17 Cluster-Server-1 crm_attribute: [51141]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:22:17 Cluster-Server-1 crm_attribute: [51141]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:22:17 Cluster-Server-1 crm_attribute: [51141]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:22:17 Cluster-Server-1 crm_attribute: [51141]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:22:17 Cluster-Server-1 crm_attribute: [51141]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:22:17 Cluster-Server-1 crm_attribute: [51141]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:22:17 Cluster-Server-1 crm_attribute: [51141]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:22:17 Cluster-Server-1 crm_attribute: [51141]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:22:17 Cluster-Server-1 crm_attribute: [51141]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:22:17 Cluster-Server-1 crm_attribute: [51141]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:22:17 Cluster-Server-1 crm_attribute: [51141]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:22:17 Cluster-Server-1 crm_attribute: [51141]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:22:17 Cluster-Server-1 crm_attribute: [51141]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:22:17 Cluster-Server-1 crm_attribute: [51141]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:22:17 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:22:17 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[51111]:	2012/09/10_15:22:17 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[51111]:	2012/09/10_15:22:17 DEBUG: drive: Command output: 
Sep 10 15:22:17 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:22:17 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 51111 exited with return code 8
Sep 10 15:22:21 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 51148)
Sep 10 15:22:23 Cluster-Server-1 attrd_updater: [51166]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:22:23 Cluster-Server-1 attrd_updater: [51166]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:22:23 Cluster-Server-1 attrd_updater: [51166]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:22:23 Cluster-Server-1 attrd_updater: [51166]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:22:23 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:22:23 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:22:23 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 51148 exited with return code 0
Sep 10 15:22:27 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 51167)
drbd(p_Device_drive:0)[51167]:	2012/09/10_15:22:27 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:22:27 Cluster-Server-1 crm_attribute: [51197]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:22:27 Cluster-Server-1 crm_attribute: [51197]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:22:27 Cluster-Server-1 crm_attribute: [51197]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:22:27 Cluster-Server-1 crm_attribute: [51197]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:22:27 Cluster-Server-1 crm_attribute: [51197]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:22:27 Cluster-Server-1 crm_attribute: [51197]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:22:27 Cluster-Server-1 crm_attribute: [51197]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:22:27 Cluster-Server-1 crm_attribute: [51197]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:22:27 Cluster-Server-1 crm_attribute: [51197]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:22:27 Cluster-Server-1 crm_attribute: [51197]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:22:27 Cluster-Server-1 crm_attribute: [51197]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:22:27 Cluster-Server-1 crm_attribute: [51197]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:22:27 Cluster-Server-1 crm_attribute: [51197]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:22:27 Cluster-Server-1 crm_attribute: [51197]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:22:27 Cluster-Server-1 crm_attribute: [51197]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:22:27 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:22:27 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[51167]:	2012/09/10_15:22:27 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[51167]:	2012/09/10_15:22:27 DEBUG: drive: Command output: 
Sep 10 15:22:27 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:22:27 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 51167 exited with return code 8
Sep 10 15:22:33 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 51204)
Sep 10 15:22:35 Cluster-Server-1 attrd_updater: [51222]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:22:35 Cluster-Server-1 attrd_updater: [51222]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:22:35 Cluster-Server-1 attrd_updater: [51222]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:22:35 Cluster-Server-1 attrd_updater: [51222]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:22:35 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:22:35 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:22:35 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 51204 exited with return code 0
Sep 10 15:22:37 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 51223)
drbd(p_Device_drive:0)[51223]:	2012/09/10_15:22:37 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:22:37 Cluster-Server-1 crm_attribute: [51253]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:22:37 Cluster-Server-1 crm_attribute: [51253]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:22:37 Cluster-Server-1 crm_attribute: [51253]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:22:37 Cluster-Server-1 crm_attribute: [51253]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:22:37 Cluster-Server-1 crm_attribute: [51253]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:22:37 Cluster-Server-1 crm_attribute: [51253]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:22:37 Cluster-Server-1 crm_attribute: [51253]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:22:37 Cluster-Server-1 crm_attribute: [51253]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:22:37 Cluster-Server-1 crm_attribute: [51253]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:22:37 Cluster-Server-1 crm_attribute: [51253]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:22:37 Cluster-Server-1 crm_attribute: [51253]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:22:37 Cluster-Server-1 crm_attribute: [51253]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:22:37 Cluster-Server-1 crm_attribute: [51253]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:22:37 Cluster-Server-1 crm_attribute: [51253]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:22:37 Cluster-Server-1 crm_attribute: [51253]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:22:37 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:22:37 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[51223]:	2012/09/10_15:22:37 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[51223]:	2012/09/10_15:22:37 DEBUG: drive: Command output: 
Sep 10 15:22:37 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:22:37 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 51223 exited with return code 8
Sep 10 15:22:45 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_iSCSI_Daemon:0 monitor[5] (pid 51332)
Sep 10 15:22:45 Cluster-Server-1 lrmd: [48712]: info: operation monitor[5] on p_iSCSI_Daemon:0 for client 48715: pid 51332 exited with return code 0
Sep 10 15:22:45 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_NFS_Server:0 monitor[8] (pid 51336)
Sep 10 15:22:45 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_NFS_Server:0:monitor:stdout) nfsd running

Sep 10 15:22:45 Cluster-Server-1 lrmd: [48712]: info: operation monitor[8] on p_NFS_Server:0 for client 48715: pid 51336 exited with return code 0
Sep 10 15:22:45 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 51338)
Sep 10 15:22:47 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 51357)
drbd(p_Device_drive:0)[51357]:	2012/09/10_15:22:47 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:22:47 Cluster-Server-1 crm_attribute: [51387]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:22:47 Cluster-Server-1 crm_attribute: [51387]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:22:47 Cluster-Server-1 crm_attribute: [51387]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:22:47 Cluster-Server-1 crm_attribute: [51387]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:22:47 Cluster-Server-1 crm_attribute: [51387]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:22:47 Cluster-Server-1 crm_attribute: [51387]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:22:47 Cluster-Server-1 crm_attribute: [51387]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:22:47 Cluster-Server-1 crm_attribute: [51387]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:22:47 Cluster-Server-1 crm_attribute: [51387]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:22:47 Cluster-Server-1 crm_attribute: [51387]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:22:47 Cluster-Server-1 crm_attribute: [51387]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:22:47 Cluster-Server-1 crm_attribute: [51387]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:22:47 Cluster-Server-1 crm_attribute: [51387]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:22:47 Cluster-Server-1 crm_attribute: [51387]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:22:47 Cluster-Server-1 crm_attribute: [51387]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:22:47 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:22:47 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[51357]:	2012/09/10_15:22:47 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[51357]:	2012/09/10_15:22:47 DEBUG: drive: Command output: 
Sep 10 15:22:47 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:22:47 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 51357 exited with return code 8
Sep 10 15:22:47 Cluster-Server-1 attrd_updater: [51396]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:22:47 Cluster-Server-1 attrd_updater: [51396]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:22:47 Cluster-Server-1 attrd_updater: [51396]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:22:47 Cluster-Server-1 attrd_updater: [51396]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:22:47 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:22:47 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:22:47 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 51338 exited with return code 0
Sep 10 15:22:57 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 51457)
Sep 10 15:22:57 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 51460)
drbd(p_Device_drive:0)[51457]:	2012/09/10_15:22:57 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:22:57 Cluster-Server-1 crm_attribute: [51503]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:22:57 Cluster-Server-1 crm_attribute: [51503]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:22:57 Cluster-Server-1 crm_attribute: [51503]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:22:57 Cluster-Server-1 crm_attribute: [51503]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:22:57 Cluster-Server-1 crm_attribute: [51503]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:22:57 Cluster-Server-1 crm_attribute: [51503]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:22:57 Cluster-Server-1 crm_attribute: [51503]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:22:57 Cluster-Server-1 crm_attribute: [51503]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:22:57 Cluster-Server-1 crm_attribute: [51503]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:22:57 Cluster-Server-1 crm_attribute: [51503]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:22:57 Cluster-Server-1 crm_attribute: [51503]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:22:57 Cluster-Server-1 crm_attribute: [51503]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:22:57 Cluster-Server-1 crm_attribute: [51503]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:22:57 Cluster-Server-1 crm_attribute: [51503]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:22:57 Cluster-Server-1 crm_attribute: [51503]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:22:57 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:22:57 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[51457]:	2012/09/10_15:22:57 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[51457]:	2012/09/10_15:22:57 DEBUG: drive: Command output: 
Sep 10 15:22:57 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:22:57 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 51457 exited with return code 8
Sep 10 15:22:59 Cluster-Server-1 attrd_updater: [51512]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:22:59 Cluster-Server-1 attrd_updater: [51512]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:22:59 Cluster-Server-1 attrd_updater: [51512]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:22:59 Cluster-Server-1 attrd_updater: [51512]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:22:59 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:22:59 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:22:59 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 51460 exited with return code 0
Sep 10 15:23:07 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 51513)
drbd(p_Device_drive:0)[51513]:	2012/09/10_15:23:07 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:23:07 Cluster-Server-1 crm_attribute: [51543]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:23:07 Cluster-Server-1 crm_attribute: [51543]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:23:07 Cluster-Server-1 crm_attribute: [51543]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:23:07 Cluster-Server-1 crm_attribute: [51543]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:23:07 Cluster-Server-1 crm_attribute: [51543]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:23:07 Cluster-Server-1 crm_attribute: [51543]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:23:07 Cluster-Server-1 crm_attribute: [51543]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:23:07 Cluster-Server-1 crm_attribute: [51543]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:23:07 Cluster-Server-1 crm_attribute: [51543]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:23:07 Cluster-Server-1 crm_attribute: [51543]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:23:07 Cluster-Server-1 crm_attribute: [51543]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:23:07 Cluster-Server-1 crm_attribute: [51543]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:23:07 Cluster-Server-1 crm_attribute: [51543]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:23:07 Cluster-Server-1 crm_attribute: [51543]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:23:07 Cluster-Server-1 crm_attribute: [51543]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:23:07 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:23:07 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[51513]:	2012/09/10_15:23:07 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[51513]:	2012/09/10_15:23:07 DEBUG: drive: Command output: 
Sep 10 15:23:07 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:23:07 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 51513 exited with return code 8
Sep 10 15:23:09 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 51551)
Sep 10 15:23:11 Cluster-Server-1 attrd_updater: [51570]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:23:11 Cluster-Server-1 attrd_updater: [51570]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:23:11 Cluster-Server-1 attrd_updater: [51570]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:23:11 Cluster-Server-1 attrd_updater: [51570]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:23:11 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:23:11 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:23:11 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 51551 exited with return code 0
Sep 10 15:23:15 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_iSCSI_Daemon:0 monitor[5] (pid 51573)
Sep 10 15:23:15 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_NFS_Server:0 monitor[8] (pid 51576)
Sep 10 15:23:15 Cluster-Server-1 lrmd: [48712]: info: operation monitor[5] on p_iSCSI_Daemon:0 for client 48715: pid 51573 exited with return code 0
Sep 10 15:23:15 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_NFS_Server:0:monitor:stdout) nfsd running

Sep 10 15:23:15 Cluster-Server-1 lrmd: [48712]: info: operation monitor[8] on p_NFS_Server:0 for client 48715: pid 51576 exited with return code 0
Sep 10 15:23:17 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 51579)
drbd(p_Device_drive:0)[51579]:	2012/09/10_15:23:17 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:23:17 Cluster-Server-1 crm_attribute: [51609]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:23:17 Cluster-Server-1 crm_attribute: [51609]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:23:17 Cluster-Server-1 crm_attribute: [51609]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:23:17 Cluster-Server-1 crm_attribute: [51609]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:23:17 Cluster-Server-1 crm_attribute: [51609]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:23:17 Cluster-Server-1 crm_attribute: [51609]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:23:17 Cluster-Server-1 crm_attribute: [51609]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:23:17 Cluster-Server-1 crm_attribute: [51609]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:23:17 Cluster-Server-1 crm_attribute: [51609]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:23:17 Cluster-Server-1 crm_attribute: [51609]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:23:17 Cluster-Server-1 crm_attribute: [51609]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:23:17 Cluster-Server-1 crm_attribute: [51609]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:23:17 Cluster-Server-1 crm_attribute: [51609]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:23:17 Cluster-Server-1 crm_attribute: [51609]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:23:17 Cluster-Server-1 crm_attribute: [51609]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:23:17 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:23:17 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[51579]:	2012/09/10_15:23:17 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[51579]:	2012/09/10_15:23:17 DEBUG: drive: Command output: 
Sep 10 15:23:17 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:23:17 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 51579 exited with return code 8
Sep 10 15:23:21 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 51618)
Sep 10 15:23:23 Cluster-Server-1 attrd_updater: [51636]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:23:23 Cluster-Server-1 attrd_updater: [51636]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:23:23 Cluster-Server-1 attrd_updater: [51636]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:23:23 Cluster-Server-1 attrd_updater: [51636]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:23:23 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:23:23 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:23:23 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 51618 exited with return code 0
Sep 10 15:23:27 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 51637)
drbd(p_Device_drive:0)[51637]:	2012/09/10_15:23:27 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:23:27 Cluster-Server-1 crm_attribute: [51667]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:23:27 Cluster-Server-1 crm_attribute: [51667]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:23:27 Cluster-Server-1 crm_attribute: [51667]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:23:27 Cluster-Server-1 crm_attribute: [51667]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:23:27 Cluster-Server-1 crm_attribute: [51667]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:23:27 Cluster-Server-1 crm_attribute: [51667]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:23:27 Cluster-Server-1 crm_attribute: [51667]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:23:27 Cluster-Server-1 crm_attribute: [51667]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:23:27 Cluster-Server-1 crm_attribute: [51667]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:23:27 Cluster-Server-1 crm_attribute: [51667]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:23:27 Cluster-Server-1 crm_attribute: [51667]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:23:27 Cluster-Server-1 crm_attribute: [51667]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:23:27 Cluster-Server-1 crm_attribute: [51667]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:23:27 Cluster-Server-1 crm_attribute: [51667]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:23:27 Cluster-Server-1 crm_attribute: [51667]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:23:27 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:23:27 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[51637]:	2012/09/10_15:23:27 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[51637]:	2012/09/10_15:23:27 DEBUG: drive: Command output: 
Sep 10 15:23:27 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:23:27 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 51637 exited with return code 8
Sep 10 15:23:33 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 51939)
Sep 10 15:23:35 Cluster-Server-1 attrd_updater: [51957]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:23:35 Cluster-Server-1 attrd_updater: [51957]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:23:35 Cluster-Server-1 attrd_updater: [51957]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:23:35 Cluster-Server-1 attrd_updater: [51957]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:23:35 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:23:35 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:23:35 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 51939 exited with return code 0
Sep 10 15:23:37 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 51958)
drbd(p_Device_drive:0)[51958]:	2012/09/10_15:23:37 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:23:37 Cluster-Server-1 crm_attribute: [51988]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:23:37 Cluster-Server-1 crm_attribute: [51988]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:23:37 Cluster-Server-1 crm_attribute: [51988]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:23:37 Cluster-Server-1 crm_attribute: [51988]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:23:37 Cluster-Server-1 crm_attribute: [51988]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:23:37 Cluster-Server-1 crm_attribute: [51988]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:23:37 Cluster-Server-1 crm_attribute: [51988]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:23:37 Cluster-Server-1 crm_attribute: [51988]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:23:37 Cluster-Server-1 crm_attribute: [51988]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:23:37 Cluster-Server-1 crm_attribute: [51988]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:23:37 Cluster-Server-1 crm_attribute: [51988]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:23:37 Cluster-Server-1 crm_attribute: [51988]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:23:37 Cluster-Server-1 crm_attribute: [51988]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:23:37 Cluster-Server-1 crm_attribute: [51988]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:23:37 Cluster-Server-1 crm_attribute: [51988]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:23:37 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:23:37 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[51958]:	2012/09/10_15:23:37 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[51958]:	2012/09/10_15:23:37 DEBUG: drive: Command output: 
Sep 10 15:23:37 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:23:37 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 51958 exited with return code 8
Sep 10 15:23:42 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [52235] registered
Sep 10 15:23:42 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:52235] disconnected.
Sep 10 15:23:42 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:52235] is unregistered
Sep 10 15:23:42 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [52237] registered
Sep 10 15:23:42 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:52237] disconnected.
Sep 10 15:23:42 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:52237] is unregistered
Sep 10 15:23:42 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [52239] registered
Sep 10 15:23:42 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:52239] disconnected.
Sep 10 15:23:42 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:52239] is unregistered
Sep 10 15:23:42 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [52241] registered
Sep 10 15:23:42 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:52241] disconnected.
Sep 10 15:23:42 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:52241] is unregistered
Sep 10 15:23:42 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [52250] registered
Sep 10 15:23:42 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:52250] disconnected.
Sep 10 15:23:42 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:52250] is unregistered
Sep 10 15:23:42 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [52259] registered
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:52259] disconnected.
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:52259] is unregistered
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [52266] registered
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:52266] disconnected.
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:52266] is unregistered
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [52273] registered
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:52273] disconnected.
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:52273] is unregistered
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_COUNT
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: crm_uptime: Current CPU usage is: 0s, 40000us
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: crm_compare_age: Loose: 40000 vs 170000 (usec)
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: info: do_election_count_vote: Election 7 (owner: Cluster-Server-2) lost: vote from Cluster-Server-2 (Uptime)
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: update_dc: Unset DC. Was Cluster-Server-2
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_CHECK
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: do_election_check: Ignore election check: we not in an election
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_PENDING: [ state=S_NOT_DC cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_LOG   
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: do_log: FSA: Input I_PENDING from do_election_count_vote() received in state S_NOT_DC
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: notice: do_state_transition: State transition S_NOT_DC -> S_PENDING [ input=I_PENDING cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_INTEGRATE_TIMER_STOP
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_FINALIZE_TIMER_STOP
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_START
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: crm_timer_start: Started Election Trigger (I_DC_TIMEOUT:20000ms), src=37
Sep 10 15:23:43 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for cib_apply_diff op
Sep 10 15:23:43 Cluster-Server-1 cib: [48709]: debug: Forking temp process write_cib_contents
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: handle_request: Raising I_JOIN_OFFER: join-5
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_JOIN_OFFER: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ]
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_CL_JOIN_REQUEST
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: info: update_dc: Set DC to Cluster-Server-2 (3.0.6)
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_offer_respond: do_cl_join_offer_respond added action A_DC_TIMER_STOP to the FSA
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:23:43 Cluster-Server-1 cib: [48709]: info: Managed write_cib_contents process 52285 exited with return code 0.
Sep 10 15:23:43 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:1'] does not exist
Sep 10 15:23:43 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update -22 for master-p_Device_drive:1=(null) passed
Sep 10 15:23:43 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:23:43 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: join_query_callback: Respond to join offer join-5
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: join_query_callback: Acknowledging Cluster-Server-2 as our DC
Sep 10 15:23:43 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:23:43 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:23:43 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 43 for probe_complete=true passed
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: handle_request: Raising I_JOIN_RESULT: join-5
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_JOIN_RESULT: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ]
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_CL_JOIN_RESULT
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_finalize_respond: Confirming join join-5: join_ack_nack
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_NFS_Server:0 after complete start op (interval=0)
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_NFS_Server:0 after complete monitor op (interval=30000)
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_PingD:0 after complete monitor op (interval=0)
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_PingD:0 after complete monitor op (interval=10000)
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce LVM_drive after complete monitor op (interval=0)
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_Device_drive:0 after complete monitor op (interval=0)
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_Device_drive:0 after complete monitor op (interval=10000)
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_iSCSI_Daemon:0 after complete monitor op (interval=0)
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_iSCSI_Daemon:0 after complete monitor op (interval=30000)
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_finalize_respond: join-5: Join complete.  Sending local LRM status to Cluster-Server-2
Sep 10 15:23:43 Cluster-Server-1 attrd: [48713]: notice: attrd_local_callback: Sending full refresh (origin=crmd)
Sep 10 15:23:43 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: master-p_Device_drive:0 (10000)
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: attrd_update_delegate: Sent update: (null)=(null) for localhost
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_NOT_DC: [ state=S_PENDING cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_LOG   
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: do_log: FSA: Input I_NOT_DC from do_cl_join_finalize_respond() received in state S_PENDING
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: notice: do_state_transition: State transition S_PENDING -> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_INTEGRATE_TIMER_STOP
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_FINALIZE_TIMER_STOP
Sep 10 15:23:43 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for cib_replace op
Sep 10 15:23:43 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:0'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[3])
Sep 10 15:23:43 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-master-p_Device_drive.0" name="master-p_Device_drive:0" value="10000" />
Sep 10 15:23:43 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
Sep 10 15:23:43 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:23:43 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:23:43 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: pingd (100)
Sep 10 15:23:43 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:1'] does not exist
Sep 10 15:23:43 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update -22 for master-p_Device_drive:1=(null) passed
Sep 10 15:23:43 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:23:43 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:23:43 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:23:43 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:23:43 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:23:43 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:23:43 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 45 for pingd=100 passed
Sep 10 15:23:43 Cluster-Server-1 cib: [48709]: debug: Forking temp process write_cib_contents
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: debug: on_msg_add_rsc:client [48715] adds resource Target_iscsi1
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=12:7:7:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=Target_iscsi1_monitor_0
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc Target_iscsi1
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[12] on Target_iscsi1 for client 48715, its parameters: crm_feature_set=[3.0.6] iqn=[iqn.2005-07.com.example:vdisk.iscsi1] CRM_meta_timeout=[20000]  to the operation list.
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: info: rsc:Target_iscsi1 probe[12] (pid 52289)
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: debug: on_msg_add_rsc:client [48715] adds resource Lun_iscsi1
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=13:7:7:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=Lun_iscsi1_monitor_0
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc Lun_iscsi1
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[13] on Lun_iscsi1 for client 48715, its parameters: path=[/dev/drive-CSD/iscsi1_iSCSI] crm_feature_set=[3.0.6] lun=[0] handler=[vdisk_blockio] device_name=[iscsi1] CRM_meta_timeout=[20000] target_iqn=[iqn.2005-07.com.example:vdisk.iscsi1]  to the operation list.
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: info: rsc:Lun_iscsi1 probe[13] (pid 52290)
Sep 10 15:23:43 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 47 for master-p_Device_drive:0=10000 passed
Sep 10 15:23:43 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 49 for probe_complete=true passed
Sep 10 15:23:43 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 52 for probe_complete=true passed
Sep 10 15:23:43 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 54 for pingd=100 passed
Sep 10 15:23:43 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 56 for pingd=100 passed
SCSTTarget(Target_iscsi1)[52289]:	2012/09/10_15:23:43 DEBUG: Target_iscsi1 monitor : 7
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: WARN: Managed Target_iscsi1:monitor process 52289 exited with return code 7.
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: info: operation monitor[12] on Target_iscsi1 for client 48715: pid 52289 exited with return code 7
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce Target_iscsi1 after complete monitor op (interval=0)
SCSTLun(Lun_iscsi1)[52290]:	2012/09/10_15:23:43 INFO: Lun_iscsi1 monitor : 7
Sep 10 15:23:43 Cluster-Server-1 cib: [48709]: info: Managed write_cib_contents process 52288 exited with return code 0.
SCSTLun(Lun_iscsi1)[52290]:	2012/09/10_15:23:43 INFO: Lun_iscsi1 monitor : 7
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: WARN: Managed Lun_iscsi1:monitor process 52290 exited with return code 7.
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: info: operation monitor[13] on Lun_iscsi1 for client 48715: pid 52290 exited with return code 7
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Target_iscsi1_monitor_0 (call=12, rc=7, cib-update=31, confirmed=true) not running
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'Target_iscsi1'
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce Lun_iscsi1 after complete monitor op (interval=0)
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Lun_iscsi1_monitor_0 (call=13, rc=7, cib-update=32, confirmed=true) not running
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'Lun_iscsi1'
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: attrd_update_delegate: Sent update: probe_complete=true for localhost
Sep 10 15:23:43 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crmd: probe_complete=true
Sep 10 15:23:43 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: true, Current: true, Stored: true
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=74:7:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=Target_iscsi1_start_0
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc Target_iscsi1
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation start[14] on Target_iscsi1 for client 48715, its parameters: crm_feature_set=[3.0.6] CRM_meta_name=[start] iqn=[iqn.2005-07.com.example:vdisk.iscsi1] CRM_meta_timeout=[240000]  to the operation list.
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: info: rsc:Target_iscsi1 start[14] (pid 52315)
SCSTTarget(Target_iscsi1)[52315]:	2012/09/10_15:23:43 INFO: target iqn.2005-07.com.example:vdisk.iscsi1: Starting...
SCSTTarget(Target_iscsi1)[52315]:	2012/09/10_15:23:43 INFO: target iqn.2005-07.com.example:vdisk.iscsi1: Starting...
SCSTTarget(Target_iscsi1)[52315]:	2012/09/10_15:23:43 INFO: Enabling target iqn.2005-07.com.example:vdisk.iscsi1
SCSTTarget(Target_iscsi1)[52315]:	2012/09/10_15:23:43 INFO: Enabling target iqn.2005-07.com.example:vdisk.iscsi1
SCSTTarget(Target_iscsi1)[52315]:	2012/09/10_15:23:43 DEBUG: SCST target iqn.2005-07.com.example:vdisk.iscsi1: Started.
SCSTTarget(Target_iscsi1)[52315]:	2012/09/10_15:23:43 DEBUG: Target_iscsi1 start : 0
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: info: Managed Target_iscsi1:start process 52315 exited with return code 0.
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: info: operation start[14] on Target_iscsi1 for client 48715: pid 52315 exited with return code 0
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce Target_iscsi1 after complete start op (interval=0)
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Target_iscsi1_start_0 (call=14, rc=0, cib-update=33, confirmed=true) ok
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending start op to history for 'Target_iscsi1'
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=75:7:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=Target_iscsi1_monitor_10000
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[15] on Target_iscsi1 for client 48715, its parameters: crm_feature_set=[3.0.6] CRM_meta_name=[monitor] iqn=[iqn.2005-07.com.example:vdisk.iscsi1] CRM_meta_timeout=[60000] CRM_meta_interval=[10000]  to the operation list.
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: info: rsc:Target_iscsi1 monitor[15] (pid 52335)
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=76:7:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=Lun_iscsi1_start_0
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc Lun_iscsi1
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation start[16] on Lun_iscsi1 for client 48715, its parameters: handler=[vdisk_blockio] CRM_meta_timeout=[60000] CRM_meta_name=[start] target_iqn=[iqn.2005-07.com.example:vdisk.iscsi1] path=[/dev/drive-CSD/iscsi1_iSCSI] crm_feature_set=[3.0.6] lun=[0] device_name=[iscsi1]  to the operation list.
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: info: rsc:Lun_iscsi1 start[16] (pid 52336)
SCSTTarget(Target_iscsi1)[52335]:	2012/09/10_15:23:43 DEBUG: Target_iscsi1 monitor : 0
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: info: Managed Target_iscsi1:monitor process 52335 exited with return code 0.
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: info: operation monitor[15] on Target_iscsi1 for client 48715: pid 52335 exited with return code 0
SCSTLun(Lun_iscsi1)[52336]:	2012/09/10_15:23:43 INFO: Disabling target iqn.2005-07.com.example:vdisk.iscsi1
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce Target_iscsi1 after complete monitor op (interval=10000)
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Target_iscsi1_monitor_10000 (call=15, rc=0, cib-update=34, confirmed=false) ok
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'Target_iscsi1'
SCSTLun(Lun_iscsi1)[52336]:	2012/09/10_15:23:43 INFO: Disabling target iqn.2005-07.com.example:vdisk.iscsi1
SCSTLun(Lun_iscsi1)[52336]:	2012/09/10_15:23:43 INFO: Starting lun 0 on target iqn.2005-07.com.example:vdisk.iscsi1
SCSTLun(Lun_iscsi1)[52336]:	2012/09/10_15:23:43 INFO: Starting lun 0 on target iqn.2005-07.com.example:vdisk.iscsi1
SCSTLun(Lun_iscsi1)[52336]:	2012/09/10_15:23:43 INFO: Opening device iscsi1, target iqn.2005-07.com.example:vdisk.iscsi1
SCSTLun(Lun_iscsi1)[52336]:	2012/09/10_15:23:43 INFO: Opening device iscsi1, target iqn.2005-07.com.example:vdisk.iscsi1
SCSTLun(Lun_iscsi1)[52336]:	2012/09/10_15:23:43 INFO: Adding LUN 0, device iscsi1, target iqn.2005-07.com.example:vdisk.iscsi1
SCSTLun(Lun_iscsi1)[52336]:	2012/09/10_15:23:43 INFO: Adding LUN 0, device iscsi1, target iqn.2005-07.com.example:vdisk.iscsi1
SCSTLun(Lun_iscsi1)[52336]:	2012/09/10_15:23:43 INFO: Enabling target iqn.2005-07.com.example:vdisk.iscsi1
SCSTLun(Lun_iscsi1)[52336]:	2012/09/10_15:23:43 INFO: Enabling target iqn.2005-07.com.example:vdisk.iscsi1
SCSTLun(Lun_iscsi1)[52336]:	2012/09/10_15:23:43 INFO: Started lun 0 on target iqn.2005-07.com.example:vdisk.iscsi1
SCSTLun(Lun_iscsi1)[52336]:	2012/09/10_15:23:43 INFO: Started lun 0 on target iqn.2005-07.com.example:vdisk.iscsi1
SCSTLun(Lun_iscsi1)[52336]:	2012/09/10_15:23:43 INFO: Lun_iscsi1 start : 0
SCSTLun(Lun_iscsi1)[52336]:	2012/09/10_15:23:43 INFO: Lun_iscsi1 start : 0
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: info: Managed Lun_iscsi1:start process 52336 exited with return code 0.
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: info: operation start[16] on Lun_iscsi1 for client 48715: pid 52336 exited with return code 0
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce Lun_iscsi1 after complete start op (interval=0)
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Lun_iscsi1_start_0 (call=16, rc=0, cib-update=35, confirmed=true) ok
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending start op to history for 'Lun_iscsi1'
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=77:7:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=Lun_iscsi1_monitor_10000
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[17] on Lun_iscsi1 for client 48715, its parameters: handler=[vdisk_blockio] CRM_meta_timeout=[20000] CRM_meta_name=[monitor] target_iqn=[iqn.2005-07.com.example:vdisk.iscsi1] path=[/dev/drive-CSD/iscsi1_iSCSI] crm_feature_set=[3.0.6] CRM_meta_interval=[10000] lun=[0] device_name=[iscsi1]  to the operation list.
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: info: rsc:Lun_iscsi1 monitor[17] (pid 52383)
SCSTLun(Lun_iscsi1)[52383]:	2012/09/10_15:23:43 INFO: Lun_iscsi1 monitor : 0
SCSTLun(Lun_iscsi1)[52383]:	2012/09/10_15:23:43 INFO: Lun_iscsi1 monitor : 0
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: info: Managed Lun_iscsi1:monitor process 52383 exited with return code 0.
Sep 10 15:23:43 Cluster-Server-1 lrmd: [48712]: info: operation monitor[17] on Lun_iscsi1 for client 48715: pid 52383 exited with return code 0
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce Lun_iscsi1 after complete monitor op (interval=10000)
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Lun_iscsi1_monitor_10000 (call=17, rc=0, cib-update=36, confirmed=false) ok
Sep 10 15:23:43 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'Lun_iscsi1'
Sep 10 15:23:45 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_iSCSI_Daemon:0 monitor[5] (pid 52401)
Sep 10 15:23:45 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_NFS_Server:0 monitor[8] (pid 52402)
Sep 10 15:23:45 Cluster-Server-1 lrmd: [48712]: info: operation monitor[8] on p_NFS_Server:0 for client 48715: pid 52402 exited with return code 0
Sep 10 15:23:45 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_NFS_Server:0:monitor:stdout) nfsd running

Sep 10 15:23:45 Cluster-Server-1 lrmd: [48712]: info: operation monitor[5] on p_iSCSI_Daemon:0 for client 48715: pid 52401 exited with return code 0
Sep 10 15:23:45 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 52407)
Sep 10 15:23:47 Cluster-Server-1 attrd_updater: [52440]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:23:47 Cluster-Server-1 attrd_updater: [52440]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:23:47 Cluster-Server-1 attrd_updater: [52440]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:23:47 Cluster-Server-1 attrd_updater: [52440]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:23:47 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:23:47 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:23:47 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 52407 exited with return code 0
Sep 10 15:23:47 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 52441)
drbd(p_Device_drive:0)[52441]:	2012/09/10_15:23:47 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:23:47 Cluster-Server-1 crm_attribute: [52471]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:23:47 Cluster-Server-1 crm_attribute: [52471]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:23:47 Cluster-Server-1 crm_attribute: [52471]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:23:47 Cluster-Server-1 crm_attribute: [52471]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:23:47 Cluster-Server-1 crm_attribute: [52471]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:23:47 Cluster-Server-1 crm_attribute: [52471]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:23:47 Cluster-Server-1 crm_attribute: [52471]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:23:47 Cluster-Server-1 crm_attribute: [52471]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:23:47 Cluster-Server-1 crm_attribute: [52471]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:23:47 Cluster-Server-1 crm_attribute: [52471]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:23:47 Cluster-Server-1 crm_attribute: [52471]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:23:47 Cluster-Server-1 crm_attribute: [52471]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:23:47 Cluster-Server-1 crm_attribute: [52471]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:23:47 Cluster-Server-1 crm_attribute: [52471]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:23:47 Cluster-Server-1 crm_attribute: [52471]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:23:47 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:23:47 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[52441]:	2012/09/10_15:23:48 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[52441]:	2012/09/10_15:23:48 DEBUG: drive: Command output: 
Sep 10 15:23:48 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:23:48 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 52441 exited with return code 8
Sep 10 15:23:53 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi1 monitor[15] (pid 52482)
SCSTTarget(Target_iscsi1)[52482]:	2012/09/10_15:23:53 DEBUG: Target_iscsi1 monitor : 0
Sep 10 15:23:53 Cluster-Server-1 lrmd: [48712]: info: operation monitor[15] on Target_iscsi1 for client 48715: pid 52482 exited with return code 0
Sep 10 15:23:53 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi1 monitor[17] (pid 52488)
SCSTLun(Lun_iscsi1)[52488]:	2012/09/10_15:23:53 INFO: Lun_iscsi1 monitor : 0
SCSTLun(Lun_iscsi1)[52488]:	2012/09/10_15:23:53 INFO: Lun_iscsi1 monitor : 0
Sep 10 15:23:53 Cluster-Server-1 lrmd: [48712]: info: operation monitor[17] on Lun_iscsi1 for client 48715: pid 52488 exited with return code 0
Sep 10 15:23:57 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 52669)
Sep 10 15:23:58 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 52685)
drbd(p_Device_drive:0)[52685]:	2012/09/10_15:23:58 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:23:58 Cluster-Server-1 crm_attribute: [52715]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:23:58 Cluster-Server-1 crm_attribute: [52715]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:23:58 Cluster-Server-1 crm_attribute: [52715]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:23:58 Cluster-Server-1 crm_attribute: [52715]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:23:58 Cluster-Server-1 crm_attribute: [52715]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:23:58 Cluster-Server-1 crm_attribute: [52715]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:23:58 Cluster-Server-1 crm_attribute: [52715]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:23:58 Cluster-Server-1 crm_attribute: [52715]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:23:58 Cluster-Server-1 crm_attribute: [52715]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:23:58 Cluster-Server-1 crm_attribute: [52715]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:23:58 Cluster-Server-1 crm_attribute: [52715]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:23:58 Cluster-Server-1 crm_attribute: [52715]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:23:58 Cluster-Server-1 crm_attribute: [52715]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:23:58 Cluster-Server-1 crm_attribute: [52715]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:23:58 Cluster-Server-1 crm_attribute: [52715]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:23:58 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:23:58 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[52685]:	2012/09/10_15:23:58 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[52685]:	2012/09/10_15:23:58 DEBUG: drive: Command output: 
Sep 10 15:23:58 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:23:58 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 52685 exited with return code 8
Sep 10 15:23:59 Cluster-Server-1 attrd_updater: [52724]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:23:59 Cluster-Server-1 attrd_updater: [52724]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:23:59 Cluster-Server-1 attrd_updater: [52724]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:23:59 Cluster-Server-1 attrd_updater: [52724]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:23:59 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:23:59 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:23:59 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 52669 exited with return code 0
Sep 10 15:24:03 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi1 monitor[15] (pid 52918)
SCSTTarget(Target_iscsi1)[52918]:	2012/09/10_15:24:03 DEBUG: Target_iscsi1 monitor : 0
Sep 10 15:24:03 Cluster-Server-1 lrmd: [48712]: info: operation monitor[15] on Target_iscsi1 for client 48715: pid 52918 exited with return code 0
Sep 10 15:24:03 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi1 monitor[17] (pid 52924)
SCSTLun(Lun_iscsi1)[52924]:	2012/09/10_15:24:03 INFO: Lun_iscsi1 monitor : 0
SCSTLun(Lun_iscsi1)[52924]:	2012/09/10_15:24:03 INFO: Lun_iscsi1 monitor : 0
Sep 10 15:24:03 Cluster-Server-1 lrmd: [48712]: info: operation monitor[17] on Lun_iscsi1 for client 48715: pid 52924 exited with return code 0
Sep 10 15:24:08 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 52932)
drbd(p_Device_drive:0)[52932]:	2012/09/10_15:24:08 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:24:08 Cluster-Server-1 crm_attribute: [52962]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:24:08 Cluster-Server-1 crm_attribute: [52962]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:24:08 Cluster-Server-1 crm_attribute: [52962]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:24:08 Cluster-Server-1 crm_attribute: [52962]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:24:08 Cluster-Server-1 crm_attribute: [52962]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:24:08 Cluster-Server-1 crm_attribute: [52962]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:24:08 Cluster-Server-1 crm_attribute: [52962]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:24:08 Cluster-Server-1 crm_attribute: [52962]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:24:08 Cluster-Server-1 crm_attribute: [52962]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:24:08 Cluster-Server-1 crm_attribute: [52962]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:24:08 Cluster-Server-1 crm_attribute: [52962]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:24:08 Cluster-Server-1 crm_attribute: [52962]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:24:08 Cluster-Server-1 crm_attribute: [52962]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:24:08 Cluster-Server-1 crm_attribute: [52962]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:24:08 Cluster-Server-1 crm_attribute: [52962]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:24:08 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:24:08 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[52932]:	2012/09/10_15:24:08 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[52932]:	2012/09/10_15:24:08 DEBUG: drive: Command output: 
Sep 10 15:24:08 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:24:08 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 52932 exited with return code 8
Sep 10 15:24:09 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 52969)
Sep 10 15:24:11 Cluster-Server-1 attrd_updater: [53169]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:24:11 Cluster-Server-1 attrd_updater: [53169]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:24:11 Cluster-Server-1 attrd_updater: [53169]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:24:11 Cluster-Server-1 attrd_updater: [53169]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:24:11 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:24:11 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:24:11 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 52969 exited with return code 0
Sep 10 15:24:13 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi1 monitor[15] (pid 53183)
SCSTTarget(Target_iscsi1)[53183]:	2012/09/10_15:24:13 DEBUG: Target_iscsi1 monitor : 0
Sep 10 15:24:13 Cluster-Server-1 lrmd: [48712]: info: operation monitor[15] on Target_iscsi1 for client 48715: pid 53183 exited with return code 0
Sep 10 15:24:13 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi1 monitor[17] (pid 53189)
SCSTLun(Lun_iscsi1)[53189]:	2012/09/10_15:24:13 INFO: Lun_iscsi1 monitor : 0
SCSTLun(Lun_iscsi1)[53189]:	2012/09/10_15:24:13 INFO: Lun_iscsi1 monitor : 0
Sep 10 15:24:13 Cluster-Server-1 lrmd: [48712]: info: operation monitor[17] on Lun_iscsi1 for client 48715: pid 53189 exited with return code 0
Sep 10 15:24:14 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [53288] registered
Sep 10 15:24:14 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:53288] disconnected.
Sep 10 15:24:14 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:53288] is unregistered
Sep 10 15:24:14 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [53290] registered
Sep 10 15:24:14 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:53290] disconnected.
Sep 10 15:24:14 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:53290] is unregistered
Sep 10 15:24:14 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [53292] registered
Sep 10 15:24:14 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:53292] disconnected.
Sep 10 15:24:14 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:53292] is unregistered
Sep 10 15:24:14 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [53294] registered
Sep 10 15:24:14 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:53294] disconnected.
Sep 10 15:24:14 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:53294] is unregistered
Sep 10 15:24:14 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [53303] registered
Sep 10 15:24:14 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:53303] disconnected.
Sep 10 15:24:14 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:53303] is unregistered
Sep 10 15:24:14 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [53312] registered
Sep 10 15:24:14 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:53312] disconnected.
Sep 10 15:24:14 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:53312] is unregistered
Sep 10 15:24:14 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [53319] registered
Sep 10 15:24:14 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:53319] disconnected.
Sep 10 15:24:14 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:53319] is unregistered
Sep 10 15:24:14 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [53326] registered
Sep 10 15:24:14 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:53326] disconnected.
Sep 10 15:24:14 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:53326] is unregistered
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_COUNT
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: crm_uptime: Current CPU usage is: 0s, 60000us
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: crm_compare_age: Loose: 60000 vs 210000 (usec)
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: info: do_election_count_vote: Election 8 (owner: Cluster-Server-2) lost: vote from Cluster-Server-2 (Uptime)
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: update_dc: Unset DC. Was Cluster-Server-2
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_CHECK
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: do_election_check: Ignore election check: we not in an election
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_PENDING: [ state=S_NOT_DC cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_LOG   
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: do_log: FSA: Input I_PENDING from do_election_count_vote() received in state S_NOT_DC
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: notice: do_state_transition: State transition S_NOT_DC -> S_PENDING [ input=I_PENDING cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_INTEGRATE_TIMER_STOP
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_FINALIZE_TIMER_STOP
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_START
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: crm_timer_start: Started Election Trigger (I_DC_TIMEOUT:20000ms), src=45
Sep 10 15:24:15 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for cib_apply_diff op
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: handle_request: Raising I_JOIN_OFFER: join-6
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_JOIN_OFFER: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ]
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_CL_JOIN_REQUEST
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: info: update_dc: Set DC to Cluster-Server-2 (3.0.6)
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_offer_respond: do_cl_join_offer_respond added action A_DC_TIMER_STOP to the FSA
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:24:15 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:1'] does not exist
Sep 10 15:24:15 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update -22 for master-p_Device_drive:1=(null) passed
Sep 10 15:24:15 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:24:15 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:24:15 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:24:15 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:24:15 Cluster-Server-1 cib: [48709]: debug: Forking temp process write_cib_contents
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: join_query_callback: Respond to join offer join-6
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: join_query_callback: Acknowledging Cluster-Server-2 as our DC
Sep 10 15:24:15 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 59 for probe_complete=true passed
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: handle_request: Raising I_JOIN_RESULT: join-6
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_JOIN_RESULT: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ]
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_CL_JOIN_RESULT
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_finalize_respond: Confirming join join-6: join_ack_nack
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_NFS_Server:0 after complete start op (interval=0)
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_NFS_Server:0 after complete monitor op (interval=30000)
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_PingD:0 after complete monitor op (interval=0)
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_PingD:0 after complete monitor op (interval=10000)
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce LVM_drive after complete monitor op (interval=0)
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_Device_drive:0 after complete monitor op (interval=0)
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_Device_drive:0 after complete monitor op (interval=10000)
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_iSCSI_Daemon:0 after complete monitor op (interval=0)
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_iSCSI_Daemon:0 after complete monitor op (interval=30000)
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Target_iscsi1 after complete start op (interval=0)
Sep 10 15:24:15 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 61 for pingd=100 passed
Sep 10 15:24:15 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for cib_replace op
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Target_iscsi1 after complete monitor op (interval=10000)
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Lun_iscsi1 after complete start op (interval=0)
Sep 10 15:24:15 Cluster-Server-1 cib: [48709]: info: Managed write_cib_contents process 53340 exited with return code 0.
Sep 10 15:24:15 Cluster-Server-1 cib: [48709]: debug: Forking temp process write_cib_contents
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Lun_iscsi1 after complete monitor op (interval=10000)
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_finalize_respond: join-6: Join complete.  Sending local LRM status to Cluster-Server-2
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: attrd_update_delegate: Sent update: (null)=(null) for localhost
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_NOT_DC: [ state=S_PENDING cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_LOG   
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: do_log: FSA: Input I_NOT_DC from do_cl_join_finalize_respond() received in state S_PENDING
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: notice: do_state_transition: State transition S_PENDING -> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_INTEGRATE_TIMER_STOP
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_FINALIZE_TIMER_STOP
Sep 10 15:24:15 Cluster-Server-1 attrd: [48713]: notice: attrd_local_callback: Sending full refresh (origin=crmd)
Sep 10 15:24:15 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: master-p_Device_drive:0 (10000)
Sep 10 15:24:15 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:0'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[3])
Sep 10 15:24:15 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-master-p_Device_drive.0" name="master-p_Device_drive:0" value="10000" />
Sep 10 15:24:15 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
Sep 10 15:24:15 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:24:15 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:24:15 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: pingd (100)
Sep 10 15:24:15 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:24:15 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:24:15 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 63 for master-p_Device_drive:0=10000 passed
Sep 10 15:24:15 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 65 for probe_complete=true passed
Sep 10 15:24:15 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 67 for pingd=100 passed
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: debug: on_msg_add_rsc:client [48715] adds resource Target_iscsi2
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=14:8:7:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=Target_iscsi2_monitor_0
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc Target_iscsi2
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[18] on Target_iscsi2 for client 48715, its parameters: crm_feature_set=[3.0.6] iqn=[iqn.2005-07.com.example:vdisk.iscsi2] CRM_meta_timeout=[20000]  to the operation list.
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: info: rsc:Target_iscsi2 probe[18] (pid 53352)
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: debug: on_msg_add_rsc:client [48715] adds resource Lun_iscsi2
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=15:8:7:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=Lun_iscsi2_monitor_0
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc Lun_iscsi2
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[19] on Lun_iscsi2 for client 48715, its parameters: path=[/dev/drive-CSD/iscsi2_iSCSI] crm_feature_set=[3.0.6] lun=[0] handler=[vdisk_blockio] device_name=[iscsi2] CRM_meta_timeout=[20000] target_iqn=[iqn.2005-07.com.example:vdisk.iscsi2]  to the operation list.
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: info: rsc:Lun_iscsi2 probe[19] (pid 53353)
Sep 10 15:24:15 Cluster-Server-1 cib: [48709]: info: Managed write_cib_contents process 53348 exited with return code 0.
Sep 10 15:24:15 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:1'] does not exist
Sep 10 15:24:15 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update -22 for master-p_Device_drive:1=(null) passed
Sep 10 15:24:15 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:24:15 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:24:15 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:24:15 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:24:15 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 70 for probe_complete=true passed
SCSTTarget(Target_iscsi2)[53352]:	2012/09/10_15:24:15 DEBUG: Target_iscsi2 monitor : 7
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: WARN: Managed Target_iscsi2:monitor process 53352 exited with return code 7.
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: info: operation monitor[18] on Target_iscsi2 for client 48715: pid 53352 exited with return code 7
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce Target_iscsi2 after complete monitor op (interval=0)
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Target_iscsi2_monitor_0 (call=18, rc=7, cib-update=40, confirmed=true) not running
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'Target_iscsi2'
Sep 10 15:24:15 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 72 for pingd=100 passed
SCSTLun(Lun_iscsi2)[53353]:	2012/09/10_15:24:15 INFO: Lun_iscsi2 monitor : 7
SCSTLun(Lun_iscsi2)[53353]:	2012/09/10_15:24:15 INFO: Lun_iscsi2 monitor : 7
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: WARN: Managed Lun_iscsi2:monitor process 53353 exited with return code 7.
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: info: operation monitor[19] on Lun_iscsi2 for client 48715: pid 53353 exited with return code 7
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce Lun_iscsi2 after complete monitor op (interval=0)
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Lun_iscsi2_monitor_0 (call=19, rc=7, cib-update=41, confirmed=true) not running
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'Lun_iscsi2'
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: attrd_update_delegate: Sent update: probe_complete=true for localhost
Sep 10 15:24:15 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crmd: probe_complete=true
Sep 10 15:24:15 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: true, Current: true, Stored: true
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=84:8:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=Target_iscsi2_start_0
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc Target_iscsi2
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation start[20] on Target_iscsi2 for client 48715, its parameters: crm_feature_set=[3.0.6] CRM_meta_name=[start] iqn=[iqn.2005-07.com.example:vdisk.iscsi2] CRM_meta_timeout=[240000]  to the operation list.
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: info: rsc:Target_iscsi2 start[20] (pid 53368)
SCSTTarget(Target_iscsi2)[53368]:	2012/09/10_15:24:15 INFO: target iqn.2005-07.com.example:vdisk.iscsi2: Starting...
SCSTTarget(Target_iscsi2)[53368]:	2012/09/10_15:24:15 INFO: target iqn.2005-07.com.example:vdisk.iscsi2: Starting...
SCSTTarget(Target_iscsi2)[53368]:	2012/09/10_15:24:15 INFO: Enabling target iqn.2005-07.com.example:vdisk.iscsi2
SCSTTarget(Target_iscsi2)[53368]:	2012/09/10_15:24:15 INFO: Enabling target iqn.2005-07.com.example:vdisk.iscsi2
SCSTTarget(Target_iscsi2)[53368]:	2012/09/10_15:24:15 DEBUG: SCST target iqn.2005-07.com.example:vdisk.iscsi2: Started.
SCSTTarget(Target_iscsi2)[53368]:	2012/09/10_15:24:15 DEBUG: Target_iscsi2 start : 0
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: info: Managed Target_iscsi2:start process 53368 exited with return code 0.
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: info: operation start[20] on Target_iscsi2 for client 48715: pid 53368 exited with return code 0
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce Target_iscsi2 after complete start op (interval=0)
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Target_iscsi2_start_0 (call=20, rc=0, cib-update=42, confirmed=true) ok
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending start op to history for 'Target_iscsi2'
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=85:8:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=Target_iscsi2_monitor_10000
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[21] on Target_iscsi2 for client 48715, its parameters: crm_feature_set=[3.0.6] CRM_meta_name=[monitor] iqn=[iqn.2005-07.com.example:vdisk.iscsi2] CRM_meta_timeout=[60000] CRM_meta_interval=[10000]  to the operation list.
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: info: rsc:Target_iscsi2 monitor[21] (pid 53388)
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=86:8:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=Lun_iscsi2_start_0
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc Lun_iscsi2
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation start[22] on Lun_iscsi2 for client 48715, its parameters: handler=[vdisk_blockio] CRM_meta_timeout=[60000] CRM_meta_name=[start] target_iqn=[iqn.2005-07.com.example:vdisk.iscsi2] path=[/dev/drive-CSD/iscsi2_iSCSI] crm_feature_set=[3.0.6] lun=[0] device_name=[iscsi2]  to the operation list.
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: info: rsc:Lun_iscsi2 start[22] (pid 53389)
SCSTTarget(Target_iscsi2)[53388]:	2012/09/10_15:24:15 DEBUG: Target_iscsi2 monitor : 0
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: info: Managed Target_iscsi2:monitor process 53388 exited with return code 0.
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: info: operation monitor[21] on Target_iscsi2 for client 48715: pid 53388 exited with return code 0
SCSTLun(Lun_iscsi2)[53389]:	2012/09/10_15:24:15 INFO: Disabling target iqn.2005-07.com.example:vdisk.iscsi2
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce Target_iscsi2 after complete monitor op (interval=10000)
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Target_iscsi2_monitor_10000 (call=21, rc=0, cib-update=43, confirmed=false) ok
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'Target_iscsi2'
SCSTLun(Lun_iscsi2)[53389]:	2012/09/10_15:24:15 INFO: Disabling target iqn.2005-07.com.example:vdisk.iscsi2
SCSTLun(Lun_iscsi2)[53389]:	2012/09/10_15:24:15 INFO: Starting lun 0 on target iqn.2005-07.com.example:vdisk.iscsi2
SCSTLun(Lun_iscsi2)[53389]:	2012/09/10_15:24:15 INFO: Starting lun 0 on target iqn.2005-07.com.example:vdisk.iscsi2
SCSTLun(Lun_iscsi2)[53389]:	2012/09/10_15:24:15 INFO: Opening device iscsi2, target iqn.2005-07.com.example:vdisk.iscsi2
SCSTLun(Lun_iscsi2)[53389]:	2012/09/10_15:24:15 INFO: Opening device iscsi2, target iqn.2005-07.com.example:vdisk.iscsi2
SCSTLun(Lun_iscsi2)[53389]:	2012/09/10_15:24:15 INFO: Adding LUN 0, device iscsi2, target iqn.2005-07.com.example:vdisk.iscsi2
SCSTLun(Lun_iscsi2)[53389]:	2012/09/10_15:24:15 INFO: Adding LUN 0, device iscsi2, target iqn.2005-07.com.example:vdisk.iscsi2
SCSTLun(Lun_iscsi2)[53389]:	2012/09/10_15:24:15 INFO: Enabling target iqn.2005-07.com.example:vdisk.iscsi2
SCSTLun(Lun_iscsi2)[53389]:	2012/09/10_15:24:15 INFO: Enabling target iqn.2005-07.com.example:vdisk.iscsi2
SCSTLun(Lun_iscsi2)[53389]:	2012/09/10_15:24:15 INFO: Started lun 0 on target iqn.2005-07.com.example:vdisk.iscsi2
SCSTLun(Lun_iscsi2)[53389]:	2012/09/10_15:24:15 INFO: Started lun 0 on target iqn.2005-07.com.example:vdisk.iscsi2
SCSTLun(Lun_iscsi2)[53389]:	2012/09/10_15:24:15 INFO: Lun_iscsi2 start : 0
SCSTLun(Lun_iscsi2)[53389]:	2012/09/10_15:24:15 INFO: Lun_iscsi2 start : 0
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: info: Managed Lun_iscsi2:start process 53389 exited with return code 0.
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: info: operation start[22] on Lun_iscsi2 for client 48715: pid 53389 exited with return code 0
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce Lun_iscsi2 after complete start op (interval=0)
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Lun_iscsi2_start_0 (call=22, rc=0, cib-update=44, confirmed=true) ok
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending start op to history for 'Lun_iscsi2'
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=87:8:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=Lun_iscsi2_monitor_10000
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[23] on Lun_iscsi2 for client 48715, its parameters: handler=[vdisk_blockio] CRM_meta_timeout=[20000] CRM_meta_name=[monitor] target_iqn=[iqn.2005-07.com.example:vdisk.iscsi2] path=[/dev/drive-CSD/iscsi2_iSCSI] crm_feature_set=[3.0.6] CRM_meta_interval=[10000] lun=[0] device_name=[iscsi2]  to the operation list.
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: info: rsc:Lun_iscsi2 monitor[23] (pid 53436)
SCSTLun(Lun_iscsi2)[53436]:	2012/09/10_15:24:15 INFO: Lun_iscsi2 monitor : 0
SCSTLun(Lun_iscsi2)[53436]:	2012/09/10_15:24:15 INFO: Lun_iscsi2 monitor : 0
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: info: Managed Lun_iscsi2:monitor process 53436 exited with return code 0.
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: info: operation monitor[23] on Lun_iscsi2 for client 48715: pid 53436 exited with return code 0
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce Lun_iscsi2 after complete monitor op (interval=10000)
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Lun_iscsi2_monitor_10000 (call=23, rc=0, cib-update=45, confirmed=false) ok
Sep 10 15:24:15 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'Lun_iscsi2'
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_NFS_Server:0 monitor[8] (pid 53444)
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_iSCSI_Daemon:0 monitor[5] (pid 53445)
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: info: operation monitor[8] on p_NFS_Server:0 for client 48715: pid 53444 exited with return code 0
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: info: operation monitor[5] on p_iSCSI_Daemon:0 for client 48715: pid 53445 exited with return code 0
Sep 10 15:24:15 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_NFS_Server:0:monitor:stdout) nfsd running

Sep 10 15:24:18 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 53462)
drbd(p_Device_drive:0)[53462]:	2012/09/10_15:24:18 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:24:18 Cluster-Server-1 crm_attribute: [53492]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:24:18 Cluster-Server-1 crm_attribute: [53492]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:24:18 Cluster-Server-1 crm_attribute: [53492]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:24:18 Cluster-Server-1 crm_attribute: [53492]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:24:18 Cluster-Server-1 crm_attribute: [53492]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:24:18 Cluster-Server-1 crm_attribute: [53492]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:24:18 Cluster-Server-1 crm_attribute: [53492]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:24:18 Cluster-Server-1 crm_attribute: [53492]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:24:18 Cluster-Server-1 crm_attribute: [53492]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:24:18 Cluster-Server-1 crm_attribute: [53492]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:24:18 Cluster-Server-1 crm_attribute: [53492]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:24:18 Cluster-Server-1 crm_attribute: [53492]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:24:18 Cluster-Server-1 crm_attribute: [53492]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:24:18 Cluster-Server-1 crm_attribute: [53492]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:24:18 Cluster-Server-1 crm_attribute: [53492]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:24:18 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:24:18 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[53462]:	2012/09/10_15:24:18 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[53462]:	2012/09/10_15:24:18 DEBUG: drive: Command output: 
Sep 10 15:24:18 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:24:18 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 53462 exited with return code 8
Sep 10 15:24:21 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 53665)
Sep 10 15:24:23 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi1 monitor[15] (pid 53681)
SCSTTarget(Target_iscsi1)[53681]:	2012/09/10_15:24:23 DEBUG: Target_iscsi1 monitor : 0
Sep 10 15:24:23 Cluster-Server-1 lrmd: [48712]: info: operation monitor[15] on Target_iscsi1 for client 48715: pid 53681 exited with return code 0
Sep 10 15:24:23 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi1 monitor[17] (pid 53687)
SCSTLun(Lun_iscsi1)[53687]:	2012/09/10_15:24:23 INFO: Lun_iscsi1 monitor : 0
SCSTLun(Lun_iscsi1)[53687]:	2012/09/10_15:24:23 INFO: Lun_iscsi1 monitor : 0
Sep 10 15:24:23 Cluster-Server-1 lrmd: [48712]: info: operation monitor[17] on Lun_iscsi1 for client 48715: pid 53687 exited with return code 0
Sep 10 15:24:23 Cluster-Server-1 attrd_updater: [53697]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:24:23 Cluster-Server-1 attrd_updater: [53697]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:24:23 Cluster-Server-1 attrd_updater: [53697]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:24:23 Cluster-Server-1 attrd_updater: [53697]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:24:23 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:24:23 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:24:23 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 53665 exited with return code 0
Sep 10 15:24:25 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi2 monitor[21] (pid 53700)
SCSTTarget(Target_iscsi2)[53700]:	2012/09/10_15:24:25 DEBUG: Target_iscsi2 monitor : 0
Sep 10 15:24:25 Cluster-Server-1 lrmd: [48712]: info: operation monitor[21] on Target_iscsi2 for client 48715: pid 53700 exited with return code 0
Sep 10 15:24:25 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi2 monitor[23] (pid 53706)
SCSTLun(Lun_iscsi2)[53706]:	2012/09/10_15:24:25 INFO: Lun_iscsi2 monitor : 0
SCSTLun(Lun_iscsi2)[53706]:	2012/09/10_15:24:25 INFO: Lun_iscsi2 monitor : 0
Sep 10 15:24:25 Cluster-Server-1 lrmd: [48712]: info: operation monitor[23] on Lun_iscsi2 for client 48715: pid 53706 exited with return code 0
Sep 10 15:24:28 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 53714)
drbd(p_Device_drive:0)[53714]:	2012/09/10_15:24:28 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:24:28 Cluster-Server-1 crm_attribute: [53744]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:24:28 Cluster-Server-1 crm_attribute: [53744]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:24:28 Cluster-Server-1 crm_attribute: [53744]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:24:28 Cluster-Server-1 crm_attribute: [53744]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:24:28 Cluster-Server-1 crm_attribute: [53744]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:24:28 Cluster-Server-1 crm_attribute: [53744]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:24:28 Cluster-Server-1 crm_attribute: [53744]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:24:28 Cluster-Server-1 crm_attribute: [53744]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:24:28 Cluster-Server-1 crm_attribute: [53744]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:24:28 Cluster-Server-1 crm_attribute: [53744]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:24:28 Cluster-Server-1 crm_attribute: [53744]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:24:28 Cluster-Server-1 crm_attribute: [53744]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:24:28 Cluster-Server-1 crm_attribute: [53744]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:24:28 Cluster-Server-1 crm_attribute: [53744]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:24:28 Cluster-Server-1 crm_attribute: [53744]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:24:28 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:24:28 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[53714]:	2012/09/10_15:24:28 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[53714]:	2012/09/10_15:24:28 DEBUG: drive: Command output: 
Sep 10 15:24:28 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:24:28 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 53714 exited with return code 8
Sep 10 15:24:33 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi1 monitor[15] (pid 54125)
SCSTTarget(Target_iscsi1)[54125]:	2012/09/10_15:24:33 DEBUG: Target_iscsi1 monitor : 0
Sep 10 15:24:33 Cluster-Server-1 lrmd: [48712]: info: operation monitor[15] on Target_iscsi1 for client 48715: pid 54125 exited with return code 0
Sep 10 15:24:33 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi1 monitor[17] (pid 54140)
SCSTLun(Lun_iscsi1)[54140]:	2012/09/10_15:24:33 INFO: Lun_iscsi1 monitor : 0
SCSTLun(Lun_iscsi1)[54140]:	2012/09/10_15:24:33 INFO: Lun_iscsi1 monitor : 0
Sep 10 15:24:33 Cluster-Server-1 lrmd: [48712]: info: operation monitor[17] on Lun_iscsi1 for client 48715: pid 54140 exited with return code 0
Sep 10 15:24:33 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 54161)
Sep 10 15:24:35 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi2 monitor[21] (pid 54211)
SCSTTarget(Target_iscsi2)[54211]:	2012/09/10_15:24:35 DEBUG: Target_iscsi2 monitor : 0
Sep 10 15:24:35 Cluster-Server-1 lrmd: [48712]: info: operation monitor[21] on Target_iscsi2 for client 48715: pid 54211 exited with return code 0
Sep 10 15:24:35 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi2 monitor[23] (pid 54217)
SCSTLun(Lun_iscsi2)[54217]:	2012/09/10_15:24:35 INFO: Lun_iscsi2 monitor : 0
SCSTLun(Lun_iscsi2)[54217]:	2012/09/10_15:24:35 INFO: Lun_iscsi2 monitor : 0
Sep 10 15:24:35 Cluster-Server-1 lrmd: [48712]: info: operation monitor[23] on Lun_iscsi2 for client 48715: pid 54217 exited with return code 0
Sep 10 15:24:35 Cluster-Server-1 attrd_updater: [54380]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:24:35 Cluster-Server-1 attrd_updater: [54380]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:24:35 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:24:35 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:24:35 Cluster-Server-1 attrd_updater: [54380]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:24:35 Cluster-Server-1 attrd_updater: [54380]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:24:35 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 54161 exited with return code 0
Sep 10 15:24:38 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 54461)
drbd(p_Device_drive:0)[54461]:	2012/09/10_15:24:38 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:24:38 Cluster-Server-1 crm_attribute: [54491]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:24:38 Cluster-Server-1 crm_attribute: [54491]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:24:38 Cluster-Server-1 crm_attribute: [54491]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:24:38 Cluster-Server-1 crm_attribute: [54491]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:24:38 Cluster-Server-1 crm_attribute: [54491]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:24:38 Cluster-Server-1 crm_attribute: [54491]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:24:38 Cluster-Server-1 crm_attribute: [54491]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:24:38 Cluster-Server-1 crm_attribute: [54491]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:24:38 Cluster-Server-1 crm_attribute: [54491]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:24:38 Cluster-Server-1 crm_attribute: [54491]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:24:38 Cluster-Server-1 crm_attribute: [54491]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:24:38 Cluster-Server-1 crm_attribute: [54491]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:24:38 Cluster-Server-1 crm_attribute: [54491]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:24:38 Cluster-Server-1 crm_attribute: [54491]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:24:38 Cluster-Server-1 crm_attribute: [54491]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:24:38 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:24:38 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[54461]:	2012/09/10_15:24:38 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[54461]:	2012/09/10_15:24:38 DEBUG: drive: Command output: 
Sep 10 15:24:38 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:24:38 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 54461 exited with return code 8
Sep 10 15:24:39 Cluster-Server-1 cib: [48709]: info: apply_xml_diff: Digest mis-match: expected a6b655d2314ddd0bdfa855f4ecc7ad93, calculated ddc6c83507ced705b25cf2f514760e7c
Sep 10 15:24:39 Cluster-Server-1 cib: [48709]: notice: cib_process_diff: Diff 0.9.39 -> 0.10.1 not applied to 0.9.39: Failed application of an update diff
Sep 10 15:24:40 Cluster-Server-1 cib: [48709]: info: cib_server_process_diff: Requesting re-sync from peer
Sep 10 15:24:40 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for cib_replace op
Sep 10 15:24:40 Cluster-Server-1 cib: [48709]: info: cib_replace_notify: Replaced: -1.-1.-1 -> 0.10.1 from Cluster-Server-2
Sep 10 15:24:40 Cluster-Server-1 crmd: [48715]: debug: do_cib_replaced: Updating the CIB after a replace: DC=false
Sep 10 15:24:40 Cluster-Server-1 attrd: [48713]: info: do_cib_replaced: Sending full refresh
Sep 10 15:24:40 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: master-p_Device_drive:0 (10000)
Sep 10 15:24:40 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:0'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[3])
Sep 10 15:24:40 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-master-p_Device_drive.0" name="master-p_Device_drive:0" value="10000" />
Sep 10 15:24:40 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
Sep 10 15:24:40 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:24:40 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:24:40 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: pingd (100)
Sep 10 15:24:40 Cluster-Server-1 cib: [48709]: debug: Forking temp process write_cib_contents
Sep 10 15:24:40 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:24:40 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:24:40 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 74 for master-p_Device_drive:0=10000 passed
Sep 10 15:24:40 Cluster-Server-1 cib: [54564]: ERROR: validate_cib_digest: Digest comparision failed: expected 83751b899e758f9b138d060ace084080 (/var/lib/heartbeat/crm/cib.ANRY1Q), calculated bd97ef3df10846e783bd64059be77e45
Sep 10 15:24:40 Cluster-Server-1 cib: [54564]: ERROR: retrieveCib: Checksum of /var/lib/heartbeat/crm/cib.uGGnOm failed!  Configuration contents ignored!
Sep 10 15:24:40 Cluster-Server-1 cib: [54564]: ERROR: retrieveCib: Usually this is caused by manual changes, please refer to http://clusterlabs.org/wiki/FAQ#cib_changes_detected
Sep 10 15:24:40 Cluster-Server-1 cib: [54564]: ERROR: crm_abort: write_cib_contents: Triggered fatal assert at io.c:662 : retrieveCib(tmp1, tmp2, FALSE) != NULL
Sep 10 15:24:40 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 76 for probe_complete=true passed
Sep 10 15:24:40 Cluster-Server-1 crmd: [48715]: debug: cancel_op: Cancelling op 17 for Lun_iscsi1 (Lun_iscsi1:17)
Sep 10 15:24:40 Cluster-Server-1 lrmd: [48712]: info: cancel_op: operation monitor[17] on Lun_iscsi1 for client 48715, its parameters: handler=[vdisk_blockio] CRM_meta_timeout=[20000] CRM_meta_name=[monitor] target_iqn=[iqn.2005-07.com.example:vdisk.iscsi1] path=[/dev/drive-CSD/iscsi1_iSCSI] crm_feature_set=[3.0.6] CRM_meta_interval=[10000] lun=[0] device_name=[iscsi1]  cancelled
Sep 10 15:24:40 Cluster-Server-1 lrmd: [48712]: debug: on_msg_cancel_op: operation 17 cancelled
Sep 10 15:24:40 Cluster-Server-1 crmd: [48715]: debug: cancel_op: Op 17 for Lun_iscsi1 (Lun_iscsi1:17): cancelled
Sep 10 15:24:40 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=75:9:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=Lun_iscsi1_stop_0
Sep 10 15:24:40 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation stop[24] on Lun_iscsi1 for client 48715, its parameters: crm_feature_set=[3.0.6] CRM_meta_name=[stop] CRM_meta_timeout=[240000]  to the operation list.
Sep 10 15:24:40 Cluster-Server-1 lrmd: [48712]: info: rsc:Lun_iscsi1 stop[24] (pid 54566)
Sep 10 15:24:40 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Lun_iscsi1_monitor_10000 (call=17, status=1, cib-update=0, confirmed=true) Cancelled
Sep 10 15:24:40 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'Lun_iscsi1'
Sep 10 15:24:40 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 78 for pingd=100 passed
Sep 10 15:24:40 Cluster-Server-1 cib: [48709]: WARN: Managed write_cib_contents process 54564 killed by signal 6 [SIGABRT - Abort].
Sep 10 15:24:40 Cluster-Server-1 cib: [48709]: ERROR: Managed write_cib_contents process 54564 dumped core
Sep 10 15:24:40 Cluster-Server-1 cib: [48709]: ERROR: cib_diskwrite_complete: Disk write failed: status=134, signo=6, exitcode=0
Sep 10 15:24:40 Cluster-Server-1 cib: [48709]: ERROR: cib_diskwrite_complete: Disabling disk writes after write failure
SCSTLun(Lun_iscsi1)[54566]:	2012/09/10_15:24:40 INFO: Stopping lun 0 on target iqn.2005-07.com.example:vdisk.iscsi1
SCSTLun(Lun_iscsi1)[54566]:	2012/09/10_15:24:40 INFO: Stopping lun 0 on target iqn.2005-07.com.example:vdisk.iscsi1
SCSTLun(Lun_iscsi1)[54566]:	2012/09/10_15:24:40 INFO: Disabling target iqn.2005-07.com.example:vdisk.iscsi1
SCSTLun(Lun_iscsi1)[54566]:	2012/09/10_15:24:40 INFO: Disabling target iqn.2005-07.com.example:vdisk.iscsi1
SCSTLun(Lun_iscsi1)[54566]:	2012/09/10_15:24:40 INFO: Removing LUN 0, device iscsi1, target iqn.2005-07.com.example:vdisk.iscsi1
SCSTLun(Lun_iscsi1)[54566]:	2012/09/10_15:24:40 INFO: Removing LUN 0, device iscsi1, target iqn.2005-07.com.example:vdisk.iscsi1
SCSTLun(Lun_iscsi1)[54566]:	2012/09/10_15:24:40 INFO: Closing device iscsi1
SCSTLun(Lun_iscsi1)[54566]:	2012/09/10_15:24:40 INFO: Closing device iscsi1
SCSTLun(Lun_iscsi1)[54566]:	2012/09/10_15:24:40 INFO: Enabling target iqn.2005-07.com.example:vdisk.iscsi1
SCSTLun(Lun_iscsi1)[54566]:	2012/09/10_15:24:40 INFO: Enabling target iqn.2005-07.com.example:vdisk.iscsi1
SCSTLun(Lun_iscsi1)[54566]:	2012/09/10_15:24:40 INFO: Lun_iscsi1 stop : 0
SCSTLun(Lun_iscsi1)[54566]:	2012/09/10_15:24:40 INFO: Lun_iscsi1 stop : 0
Sep 10 15:24:40 Cluster-Server-1 lrmd: [48712]: info: Managed Lun_iscsi1:stop process 54566 exited with return code 0.
Sep 10 15:24:40 Cluster-Server-1 lrmd: [48712]: info: operation stop[24] on Lun_iscsi1 for client 48715: pid 54566 exited with return code 0
Sep 10 15:24:40 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce Lun_iscsi1 after complete stop op (interval=0)
Sep 10 15:24:40 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Lun_iscsi1_stop_0 (call=24, rc=0, cib-update=46, confirmed=true) ok
Sep 10 15:24:40 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending stop op to history for 'Lun_iscsi1'
Sep 10 15:24:40 Cluster-Server-1 crmd: [48715]: debug: cancel_op: Cancelling op 15 for Target_iscsi1 (Target_iscsi1:15)
Sep 10 15:24:40 Cluster-Server-1 lrmd: [48712]: info: cancel_op: operation monitor[15] on Target_iscsi1 for client 48715, its parameters: crm_feature_set=[3.0.6] CRM_meta_name=[monitor] iqn=[iqn.2005-07.com.example:vdisk.iscsi1] CRM_meta_timeout=[60000] CRM_meta_interval=[10000]  cancelled
Sep 10 15:24:40 Cluster-Server-1 lrmd: [48712]: debug: on_msg_cancel_op: operation 15 cancelled
Sep 10 15:24:40 Cluster-Server-1 crmd: [48715]: debug: cancel_op: Op 15 for Target_iscsi1 (Target_iscsi1:15): cancelled
Sep 10 15:24:40 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=74:9:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=Target_iscsi1_stop_0
Sep 10 15:24:40 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation stop[25] on Target_iscsi1 for client 48715, its parameters: crm_feature_set=[3.0.6] CRM_meta_name=[stop] CRM_meta_timeout=[240000]  to the operation list.
Sep 10 15:24:40 Cluster-Server-1 lrmd: [48712]: info: rsc:Target_iscsi1 stop[25] (pid 54604)
Sep 10 15:24:40 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Target_iscsi1_monitor_10000 (call=15, status=1, cib-update=0, confirmed=true) Cancelled
Sep 10 15:24:40 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'Target_iscsi1'
SCSTTarget(Target_iscsi1)[54604]:	2012/09/10_15:24:40 INFO: target iqn.2005-07.com.example:vdisk.iscsi1: Stopping...
SCSTTarget(Target_iscsi1)[54604]:	2012/09/10_15:24:40 INFO: target iqn.2005-07.com.example:vdisk.iscsi1: Stopping...
SCSTTarget(Target_iscsi1)[54604]:	2012/09/10_15:24:40 INFO: disabling target iqn.2005-07.com.example:vdisk.iscsi1
SCSTTarget(Target_iscsi1)[54604]:	2012/09/10_15:24:40 INFO: disabling target iqn.2005-07.com.example:vdisk.iscsi1
SCSTTarget(Target_iscsi1)[54604]:	2012/09/10_15:24:40 INFO: deleting target iqn.2005-07.com.example:vdisk.iscsi1
SCSTTarget(Target_iscsi1)[54604]:	2012/09/10_15:24:40 INFO: deleting target iqn.2005-07.com.example:vdisk.iscsi1
SCSTTarget(Target_iscsi1)[54604]:	2012/09/10_15:24:40 INFO: target iqn.2005-07.com.example:vdisk.iscsi1: Stopped.
SCSTTarget(Target_iscsi1)[54604]:	2012/09/10_15:24:40 INFO: target iqn.2005-07.com.example:vdisk.iscsi1: Stopped.
SCSTTarget(Target_iscsi1)[54604]:	2012/09/10_15:24:40 DEBUG: Target_iscsi1 stop : 0
Sep 10 15:24:40 Cluster-Server-1 lrmd: [48712]: info: Managed Target_iscsi1:stop process 54604 exited with return code 0.
Sep 10 15:24:40 Cluster-Server-1 lrmd: [48712]: info: operation stop[25] on Target_iscsi1 for client 48715: pid 54604 exited with return code 0
Sep 10 15:24:40 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce Target_iscsi1 after complete stop op (interval=0)
Sep 10 15:24:40 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Target_iscsi1_stop_0 (call=25, rc=0, cib-update=47, confirmed=true) ok
Sep 10 15:24:40 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending stop op to history for 'Target_iscsi1'
Sep 10 15:24:42 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_resource: fail-count-Target_iscsi1=<null>
Sep 10 15:24:42 Cluster-Server-1 attrd: [48713]: info: find_hash_entry: Creating hash entry for fail-count-Target_iscsi1
Sep 10 15:24:42 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: (null), Current: (null), Stored: (null)
Sep 10 15:24:42 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_resource: fail-count-Lun_iscsi1=<null>
Sep 10 15:24:42 Cluster-Server-1 attrd: [48713]: info: find_hash_entry: Creating hash entry for fail-count-Lun_iscsi1
Sep 10 15:24:42 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: (null), Current: (null), Stored: (null)
Sep 10 15:24:42 Cluster-Server-1 cib: [48709]: info: apply_xml_diff: Digest mis-match: expected a700b90829c779c3d702f6268cbfa4cf, calculated f32705eada43e94fdc6e59e5d40ae5cf
Sep 10 15:24:42 Cluster-Server-1 cib: [48709]: notice: cib_process_diff: Diff 0.10.8 -> 0.10.9 not applied to 0.10.8: Failed application of an update diff
Sep 10 15:24:42 Cluster-Server-1 cib: [48709]: info: cib_server_process_diff: Requesting re-sync from peer
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: info: delete_resource: Removing resource Target_iscsi1 for 54695_crm_resource (internal) on Cluster-Server-1
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: lrmd_rsc_destroy: removing resource Target_iscsi1
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: delete_rsc_entry: sync: Sending delete op for Target_iscsi1
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: info: notify_deleted: Notifying 54695_crm_resource on Cluster-Server-1 that Target_iscsi1 was deleted
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: WARN: decode_transition_key: Bad UUID (crm-resource-54695) in sscanf result (3) for 0:0:crm-resource-54695
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: send_direct_ack: Updating resouce Target_iscsi1 after complete delete op (interval=60000)
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: send_direct_ack: ACK'ing resource op Target_iscsi1_delete_60000 from 0:0:crm-resource-54695: lrm_invoke-lrmd-1347283482-22
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: notify_deleted: Triggering a refresh after 54695_crm_resource deleted Target_iscsi1 from the LRM
Sep 10 15:24:42 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: cib_query: //cib/configuration/crm_config//cluster_property_set//nvpair[@name='last-lrm-refresh'] does not exist
Sep 10 15:24:42 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for /cib (/cib)
Sep 10 15:24:42 Cluster-Server-1 cib: [48709]: info: apply_xml_diff: Digest mis-match: expected 5bf27e45eabe2218eedc434e615d9900, calculated 7c9591feac24c870af9ec7a7dab5042e
Sep 10 15:24:42 Cluster-Server-1 cib: [48709]: notice: cib_process_diff: Diff 0.11.1 -> 0.11.2 not applied to 0.11.1: Failed application of an update diff
Sep 10 15:24:42 Cluster-Server-1 cib: [48709]: info: cib_server_process_diff: Requesting re-sync from peer
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: info: delete_resource: Removing resource Lun_iscsi1 for 54695_crm_resource (internal) on Cluster-Server-1
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: lrmd_rsc_destroy: removing resource Lun_iscsi1
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: delete_rsc_entry: sync: Sending delete op for Lun_iscsi1
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: info: notify_deleted: Notifying 54695_crm_resource on Cluster-Server-1 that Lun_iscsi1 was deleted
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: WARN: decode_transition_key: Bad UUID (crm-resource-54695) in sscanf result (3) for 0:0:crm-resource-54695
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: send_direct_ack: Updating resouce Lun_iscsi1 after complete delete op (interval=60000)
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: send_direct_ack: ACK'ing resource op Lun_iscsi1_delete_60000 from 0:0:crm-resource-54695: lrm_invoke-lrmd-1347283482-23
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: notify_deleted: Triggering a refresh after 54695_crm_resource deleted Lun_iscsi1 from the LRM
Sep 10 15:24:42 Cluster-Server-1 cib: [48709]: notice: cib_server_process_diff: Not applying diff 0.11.1 -> 0.11.2 (sync in progress)
Sep 10 15:24:42 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/configuration/crm_config//cluster_property_set//nvpair[@name='last-lrm-refresh'] (/cib/configuration/crm_config/cluster_property_set/nvpair[8])
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: find_nvpair_attr_delegate: Match <nvpair id="cib-bootstrap-options-last-lrm-refresh" name="last-lrm-refresh" value="1347283482" />
Sep 10 15:24:42 Cluster-Server-1 cib: [48709]: notice: cib_server_process_diff: Not applying diff 0.11.1 -> 0.11.2 (sync in progress)
Sep 10 15:24:42 Cluster-Server-1 cib: [48709]: info: cib_replace_notify: Replaced: -1.-1.-1 -> 0.11.2 from Cluster-Server-2
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: do_cib_replaced: Updating the CIB after a replace: DC=false
Sep 10 15:24:42 Cluster-Server-1 attrd: [48713]: info: do_cib_replaced: Sending full refresh
Sep 10 15:24:42 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: master-p_Device_drive:0 (10000)
Sep 10 15:24:42 Cluster-Server-1 cib: [48709]: info: apply_xml_diff: Digest mis-match: expected e587dd2fa3e5e20e2d3eacfcc7f57c03, calculated ae649f492750d75e8e2fb5f04f74e3b9
Sep 10 15:24:42 Cluster-Server-1 cib: [48709]: notice: cib_process_diff: Diff 0.11.5 -> 0.11.6 not applied to 0.11.5: Failed application of an update diff
Sep 10 15:24:42 Cluster-Server-1 cib: [48709]: info: cib_server_process_diff: Requesting re-sync from peer
Sep 10 15:24:42 Cluster-Server-1 cib: [48709]: notice: cib_server_process_diff: Not applying diff 0.11.5 -> 0.11.6 (sync in progress)
Sep 10 15:24:42 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:0'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[3])
Sep 10 15:24:42 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-master-p_Device_drive.0" name="master-p_Device_drive:0" value="10000" />
Sep 10 15:24:42 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: pingd (100)
Sep 10 15:24:42 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
Sep 10 15:24:42 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:24:42 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:24:42 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:24:42 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:24:42 Cluster-Server-1 cib: [48709]: notice: cib_server_process_diff: Not applying diff 0.11.6 -> 0.11.7 (sync in progress)
Sep 10 15:24:42 Cluster-Server-1 cib: [48709]: info: cib_replace_notify: Replaced: -1.-1.-1 -> 0.11.7 from Cluster-Server-2
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: do_cib_replaced: Updating the CIB after a replace: DC=false
Sep 10 15:24:42 Cluster-Server-1 attrd: [48713]: info: do_cib_replaced: Sending full refresh
Sep 10 15:24:42 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: master-p_Device_drive:0 (10000)
Sep 10 15:24:42 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:0'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[3])
Sep 10 15:24:42 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-master-p_Device_drive.0" name="master-p_Device_drive:0" value="10000" />
Sep 10 15:24:42 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: pingd (100)
Sep 10 15:24:42 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
Sep 10 15:24:42 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:24:42 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:24:42 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 80 for master-p_Device_drive:0=10000 passed
Sep 10 15:24:42 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 82 for probe_complete=true passed
Sep 10 15:24:42 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:24:42 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:24:42 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 84 for pingd=100 passed
Sep 10 15:24:42 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 86 for master-p_Device_drive:0=10000 passed
Sep 10 15:24:42 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 88 for probe_complete=true passed
Sep 10 15:24:42 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 90 for pingd=100 passed
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: on_msg_add_rsc:client [48715] adds resource Target_iscsi1
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=14:12:7:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=Target_iscsi1_monitor_0
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc Target_iscsi1
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[26] on Target_iscsi1 for client 48715, its parameters: crm_feature_set=[3.0.6] iqn=[iqn.2005-07.com.example:vdisk.iscsi1] CRM_meta_timeout=[20000]  to the operation list.
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: info: rsc:Target_iscsi1 probe[26] (pid 54703)
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: on_msg_add_rsc:client [48715] adds resource Lun_iscsi1
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=15:12:7:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=Lun_iscsi1_monitor_0
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc Lun_iscsi1
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[27] on Lun_iscsi1 for client 48715, its parameters: path=[/dev/drive-CSD/iscsi1_iSCSI] crm_feature_set=[3.0.6] lun=[0] handler=[vdisk_blockio] device_name=[iscsi1] CRM_meta_timeout=[20000] target_iqn=[iqn.2005-07.com.example:vdisk.iscsi1]  to the operation list.
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: info: rsc:Lun_iscsi1 probe[27] (pid 54709)
SCSTTarget(Target_iscsi1)[54703]:	2012/09/10_15:24:42 DEBUG: Target_iscsi1 monitor : 7
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: WARN: Managed Target_iscsi1:monitor process 54703 exited with return code 7.
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: info: operation monitor[26] on Target_iscsi1 for client 48715: pid 54703 exited with return code 7
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce Target_iscsi1 after complete monitor op (interval=0)
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Target_iscsi1_monitor_0 (call=26, rc=7, cib-update=57, confirmed=true) not running
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'Target_iscsi1'
SCSTLun(Lun_iscsi1)[54709]:	2012/09/10_15:24:42 INFO: Lun_iscsi1 monitor : 7
SCSTLun(Lun_iscsi1)[54709]:	2012/09/10_15:24:42 INFO: Lun_iscsi1 monitor : 7
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: WARN: Managed Lun_iscsi1:monitor process 54709 exited with return code 7.
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: info: operation monitor[27] on Lun_iscsi1 for client 48715: pid 54709 exited with return code 7
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce Lun_iscsi1 after complete monitor op (interval=0)
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Lun_iscsi1_monitor_0 (call=27, rc=7, cib-update=58, confirmed=true) not running
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'Lun_iscsi1'
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: attrd_update_delegate: Sent update: probe_complete=true for localhost
Sep 10 15:24:42 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crmd: probe_complete=true
Sep 10 15:24:42 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: true, Current: true, Stored: true
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [54765] registered
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:54765] disconnected.
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:54765] is unregistered
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [54767] registered
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:54767] disconnected.
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:54767] is unregistered
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [54769] registered
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:54769] disconnected.
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:54769] is unregistered
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [54771] registered
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:54771] disconnected.
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:54771] is unregistered
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [54780] registered
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:54780] disconnected.
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:54780] is unregistered
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [54789] registered
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:54789] disconnected.
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:54789] is unregistered
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [54796] registered
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:54796] disconnected.
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:54796] is unregistered
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [54803] registered
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:54803] disconnected.
Sep 10 15:24:42 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:54803] is unregistered
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_COUNT
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: crm_uptime: Current CPU usage is: 0s, 70000us
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: crm_compare_age: Loose: 70000 vs 300000 (usec)
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: info: do_election_count_vote: Election 9 (owner: Cluster-Server-2) lost: vote from Cluster-Server-2 (Uptime)
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: update_dc: Unset DC. Was Cluster-Server-2
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_CHECK
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: do_election_check: Ignore election check: we not in an election
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_PENDING: [ state=S_NOT_DC cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_LOG   
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: do_log: FSA: Input I_PENDING from do_election_count_vote() received in state S_NOT_DC
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: notice: do_state_transition: State transition S_NOT_DC -> S_PENDING [ input=I_PENDING cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_INTEGRATE_TIMER_STOP
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_FINALIZE_TIMER_STOP
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_START
Sep 10 15:24:42 Cluster-Server-1 crmd: [48715]: debug: crm_timer_start: Started Election Trigger (I_DC_TIMEOUT:20000ms), src=58
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: handle_request: Raising I_JOIN_OFFER: join-7
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_JOIN_OFFER: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ]
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_CL_JOIN_REQUEST
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: info: update_dc: Set DC to Cluster-Server-2 (3.0.6)
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_offer_respond: do_cl_join_offer_respond added action A_DC_TIMER_STOP to the FSA
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:24:43 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:1'] does not exist
Sep 10 15:24:43 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update -22 for master-p_Device_drive:1=(null) passed
Sep 10 15:24:43 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:24:43 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:24:43 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:24:43 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: join_query_callback: Respond to join offer join-7
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: join_query_callback: Acknowledging Cluster-Server-2 as our DC
Sep 10 15:24:43 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 93 for pingd=100 passed
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: handle_request: Raising I_JOIN_RESULT: join-7
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_JOIN_RESULT: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ]
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_CL_JOIN_RESULT
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_finalize_respond: Confirming join join-7: join_ack_nack
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_NFS_Server:0 after complete start op (interval=0)
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_NFS_Server:0 after complete monitor op (interval=30000)
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce LVM_drive after complete monitor op (interval=0)
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Lun_iscsi1 after complete monitor op (interval=0)
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_PingD:0 after complete monitor op (interval=0)
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_PingD:0 after complete monitor op (interval=10000)
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Lun_iscsi2 after complete start op (interval=0)
Sep 10 15:24:43 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 95 for probe_complete=true passed
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Lun_iscsi2 after complete monitor op (interval=10000)
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_Device_drive:0 after complete monitor op (interval=0)
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_Device_drive:0 after complete monitor op (interval=10000)
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_iSCSI_Daemon:0 after complete monitor op (interval=0)
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_iSCSI_Daemon:0 after complete monitor op (interval=30000)
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Target_iscsi2 after complete start op (interval=0)
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Target_iscsi2 after complete monitor op (interval=10000)
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Target_iscsi1 after complete monitor op (interval=0)
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_finalize_respond: join-7: Join complete.  Sending local LRM status to Cluster-Server-2
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: attrd_update_delegate: Sent update: (null)=(null) for localhost
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_NOT_DC: [ state=S_PENDING cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_LOG   
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: do_log: FSA: Input I_NOT_DC from do_cl_join_finalize_respond() received in state S_PENDING
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: notice: do_state_transition: State transition S_PENDING -> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_INTEGRATE_TIMER_STOP
Sep 10 15:24:43 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_FINALIZE_TIMER_STOP
Sep 10 15:24:43 Cluster-Server-1 attrd: [48713]: notice: attrd_local_callback: Sending full refresh (origin=crmd)
Sep 10 15:24:43 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: master-p_Device_drive:0 (10000)
Sep 10 15:24:43 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:0'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[3])
Sep 10 15:24:43 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-master-p_Device_drive.0" name="master-p_Device_drive:0" value="10000" />
Sep 10 15:24:43 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: pingd (100)
Sep 10 15:24:43 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
Sep 10 15:24:43 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:24:43 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:24:43 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:1'] does not exist
Sep 10 15:24:43 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update -22 for master-p_Device_drive:1=(null) passed
Sep 10 15:24:43 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:24:43 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:24:43 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:24:43 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:24:43 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:24:43 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:24:43 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 97 for master-p_Device_drive:0=10000 passed
Sep 10 15:24:43 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 99 for probe_complete=true passed
Sep 10 15:24:43 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 102 for pingd=100 passed
Sep 10 15:24:43 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 104 for probe_complete=true passed
Sep 10 15:24:43 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 106 for pingd=100 passed
Sep 10 15:24:45 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi2 monitor[21] (pid 54844)
Sep 10 15:24:45 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi2 monitor[23] (pid 54848)
SCSTTarget(Target_iscsi2)[54844]:	2012/09/10_15:24:45 DEBUG: Target_iscsi2 monitor : 0
Sep 10 15:24:45 Cluster-Server-1 lrmd: [48712]: info: operation monitor[21] on Target_iscsi2 for client 48715: pid 54844 exited with return code 0
SCSTLun(Lun_iscsi2)[54848]:	2012/09/10_15:24:45 INFO: Lun_iscsi2 monitor : 0
SCSTLun(Lun_iscsi2)[54848]:	2012/09/10_15:24:45 INFO: Lun_iscsi2 monitor : 0
Sep 10 15:24:45 Cluster-Server-1 lrmd: [48712]: info: operation monitor[23] on Lun_iscsi2 for client 48715: pid 54848 exited with return code 0
Sep 10 15:24:45 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_NFS_Server:0 monitor[8] (pid 54858)
Sep 10 15:24:45 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_iSCSI_Daemon:0 monitor[5] (pid 54859)
Sep 10 15:24:45 Cluster-Server-1 lrmd: [48712]: info: operation monitor[8] on p_NFS_Server:0 for client 48715: pid 54858 exited with return code 0
Sep 10 15:24:45 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_NFS_Server:0:monitor:stdout) nfsd running

Sep 10 15:24:45 Cluster-Server-1 lrmd: [48712]: info: operation monitor[5] on p_iSCSI_Daemon:0 for client 48715: pid 54859 exited with return code 0
Sep 10 15:24:45 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 54864)
Sep 10 15:24:47 Cluster-Server-1 attrd_updater: [54887]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:24:47 Cluster-Server-1 attrd_updater: [54887]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:24:47 Cluster-Server-1 attrd_updater: [54887]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:24:47 Cluster-Server-1 attrd_updater: [54887]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:24:47 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:24:47 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:24:47 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 54864 exited with return code 0
Sep 10 15:24:48 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 54888)
drbd(p_Device_drive:0)[54888]:	2012/09/10_15:24:48 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:24:48 Cluster-Server-1 crm_attribute: [54918]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:24:48 Cluster-Server-1 crm_attribute: [54918]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:24:48 Cluster-Server-1 crm_attribute: [54918]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:24:48 Cluster-Server-1 crm_attribute: [54918]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:24:48 Cluster-Server-1 crm_attribute: [54918]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:24:48 Cluster-Server-1 crm_attribute: [54918]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:24:48 Cluster-Server-1 crm_attribute: [54918]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:24:48 Cluster-Server-1 crm_attribute: [54918]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:24:48 Cluster-Server-1 crm_attribute: [54918]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:24:48 Cluster-Server-1 crm_attribute: [54918]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:24:48 Cluster-Server-1 crm_attribute: [54918]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:24:48 Cluster-Server-1 crm_attribute: [54918]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:24:48 Cluster-Server-1 crm_attribute: [54918]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:24:48 Cluster-Server-1 crm_attribute: [54918]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:24:48 Cluster-Server-1 crm_attribute: [54918]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:24:48 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:24:48 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[54888]:	2012/09/10_15:24:48 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[54888]:	2012/09/10_15:24:48 DEBUG: drive: Command output: 
Sep 10 15:24:48 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:24:48 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 54888 exited with return code 8
Sep 10 15:24:55 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi2 monitor[21] (pid 55053)
Sep 10 15:24:55 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi2 monitor[23] (pid 55054)
SCSTTarget(Target_iscsi2)[55053]:	2012/09/10_15:24:55 DEBUG: Target_iscsi2 monitor : 0
Sep 10 15:24:55 Cluster-Server-1 lrmd: [48712]: info: operation monitor[21] on Target_iscsi2 for client 48715: pid 55053 exited with return code 0
SCSTLun(Lun_iscsi2)[55054]:	2012/09/10_15:24:55 INFO: Lun_iscsi2 monitor : 0
SCSTLun(Lun_iscsi2)[55054]:	2012/09/10_15:24:55 INFO: Lun_iscsi2 monitor : 0
Sep 10 15:24:55 Cluster-Server-1 lrmd: [48712]: info: operation monitor[23] on Lun_iscsi2 for client 48715: pid 55054 exited with return code 0
Sep 10 15:24:57 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 55070)
Sep 10 15:24:58 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 55086)
drbd(p_Device_drive:0)[55086]:	2012/09/10_15:24:58 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:24:58 Cluster-Server-1 crm_attribute: [55116]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:24:58 Cluster-Server-1 crm_attribute: [55116]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:24:58 Cluster-Server-1 crm_attribute: [55116]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:24:58 Cluster-Server-1 crm_attribute: [55116]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:24:58 Cluster-Server-1 crm_attribute: [55116]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:24:58 Cluster-Server-1 crm_attribute: [55116]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:24:58 Cluster-Server-1 crm_attribute: [55116]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:24:58 Cluster-Server-1 crm_attribute: [55116]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:24:58 Cluster-Server-1 crm_attribute: [55116]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:24:58 Cluster-Server-1 crm_attribute: [55116]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:24:58 Cluster-Server-1 crm_attribute: [55116]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:24:58 Cluster-Server-1 crm_attribute: [55116]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:24:58 Cluster-Server-1 crm_attribute: [55116]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:24:58 Cluster-Server-1 crm_attribute: [55116]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:24:58 Cluster-Server-1 crm_attribute: [55116]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:24:58 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:24:58 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[55086]:	2012/09/10_15:24:58 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[55086]:	2012/09/10_15:24:58 DEBUG: drive: Command output: 
Sep 10 15:24:58 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:24:58 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 55086 exited with return code 8
Sep 10 15:24:59 Cluster-Server-1 attrd_updater: [55125]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:24:59 Cluster-Server-1 attrd_updater: [55125]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:24:59 Cluster-Server-1 attrd_updater: [55125]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:24:59 Cluster-Server-1 attrd_updater: [55125]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:24:59 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:24:59 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:24:59 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 55070 exited with return code 0
Sep 10 15:25:05 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi2 monitor[21] (pid 55379)
Sep 10 15:25:05 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi2 monitor[23] (pid 55380)
SCSTTarget(Target_iscsi2)[55379]:	2012/09/10_15:25:05 DEBUG: Target_iscsi2 monitor : 0
Sep 10 15:25:05 Cluster-Server-1 lrmd: [48712]: info: operation monitor[21] on Target_iscsi2 for client 48715: pid 55379 exited with return code 0
SCSTLun(Lun_iscsi2)[55380]:	2012/09/10_15:25:05 INFO: Lun_iscsi2 monitor : 0
SCSTLun(Lun_iscsi2)[55380]:	2012/09/10_15:25:05 INFO: Lun_iscsi2 monitor : 0
Sep 10 15:25:05 Cluster-Server-1 lrmd: [48712]: info: operation monitor[23] on Lun_iscsi2 for client 48715: pid 55380 exited with return code 0
Sep 10 15:25:08 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 55393)
drbd(p_Device_drive:0)[55393]:	2012/09/10_15:25:08 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:25:08 Cluster-Server-1 crm_attribute: [55423]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:25:08 Cluster-Server-1 crm_attribute: [55423]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:25:08 Cluster-Server-1 crm_attribute: [55423]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:25:08 Cluster-Server-1 crm_attribute: [55423]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:25:08 Cluster-Server-1 crm_attribute: [55423]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:25:08 Cluster-Server-1 crm_attribute: [55423]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:25:08 Cluster-Server-1 crm_attribute: [55423]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:25:08 Cluster-Server-1 crm_attribute: [55423]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:25:08 Cluster-Server-1 crm_attribute: [55423]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:25:08 Cluster-Server-1 crm_attribute: [55423]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:25:08 Cluster-Server-1 crm_attribute: [55423]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:25:08 Cluster-Server-1 crm_attribute: [55423]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:25:08 Cluster-Server-1 crm_attribute: [55423]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:25:08 Cluster-Server-1 crm_attribute: [55423]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:25:08 Cluster-Server-1 crm_attribute: [55423]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:25:08 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:25:08 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[55393]:	2012/09/10_15:25:08 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[55393]:	2012/09/10_15:25:08 DEBUG: drive: Command output: 
Sep 10 15:25:08 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:25:08 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 55393 exited with return code 8
Sep 10 15:25:09 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 55592)
Sep 10 15:25:12 Cluster-Server-1 attrd_updater: [55636]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:25:12 Cluster-Server-1 attrd_updater: [55636]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:25:12 Cluster-Server-1 attrd_updater: [55636]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:25:12 Cluster-Server-1 attrd_updater: [55636]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:25:12 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:25:12 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:25:12 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 55592 exited with return code 0
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [55732] registered
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:55732] disconnected.
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:55732] is unregistered
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [55734] registered
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:55734] disconnected.
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:55734] is unregistered
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [55736] registered
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:55736] disconnected.
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:55736] is unregistered
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [55738] registered
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:55738] disconnected.
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:55738] is unregistered
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [55747] registered
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:55747] disconnected.
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:55747] is unregistered
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [55756] registered
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:55756] disconnected.
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:55756] is unregistered
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [55763] registered
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:55763] disconnected.
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:55763] is unregistered
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [55770] registered
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:55770] disconnected.
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:55770] is unregistered
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_COUNT
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: crm_uptime: Current CPU usage is: 0s, 80000us
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: crm_compare_age: Loose: 80000 vs 310000 (usec)
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: info: do_election_count_vote: Election 10 (owner: Cluster-Server-2) lost: vote from Cluster-Server-2 (Uptime)
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: update_dc: Unset DC. Was Cluster-Server-2
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_CHECK
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: do_election_check: Ignore election check: we not in an election
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_PENDING: [ state=S_NOT_DC cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_LOG   
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: do_log: FSA: Input I_PENDING from do_election_count_vote() received in state S_NOT_DC
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: notice: do_state_transition: State transition S_NOT_DC -> S_PENDING [ input=I_PENDING cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_INTEGRATE_TIMER_STOP
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_FINALIZE_TIMER_STOP
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_START
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: crm_timer_start: Started Election Trigger (I_DC_TIMEOUT:20000ms), src=60
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: handle_request: Raising I_JOIN_OFFER: join-8
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_JOIN_OFFER: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ]
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_CL_JOIN_REQUEST
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: info: update_dc: Set DC to Cluster-Server-2 (3.0.6)
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_offer_respond: do_cl_join_offer_respond added action A_DC_TIMER_STOP to the FSA
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:25:14 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:1'] does not exist
Sep 10 15:25:14 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update -22 for master-p_Device_drive:1=(null) passed
Sep 10 15:25:14 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:25:14 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:25:14 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:25:14 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: join_query_callback: Respond to join offer join-8
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: join_query_callback: Acknowledging Cluster-Server-2 as our DC
Sep 10 15:25:14 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 109 for pingd=100 passed
Sep 10 15:25:14 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 111 for probe_complete=true passed
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: handle_request: Raising I_JOIN_RESULT: join-8
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_JOIN_RESULT: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ]
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_CL_JOIN_RESULT
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_finalize_respond: Confirming join join-8: join_ack_nack
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_NFS_Server:0 after complete start op (interval=0)
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_NFS_Server:0 after complete monitor op (interval=30000)
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce LVM_drive after complete monitor op (interval=0)
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Lun_iscsi1 after complete monitor op (interval=0)
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_PingD:0 after complete monitor op (interval=0)
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_PingD:0 after complete monitor op (interval=10000)
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Lun_iscsi2 after complete start op (interval=0)
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Lun_iscsi2 after complete monitor op (interval=10000)
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_Device_drive:0 after complete monitor op (interval=0)
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_Device_drive:0 after complete monitor op (interval=10000)
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_iSCSI_Daemon:0 after complete monitor op (interval=0)
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_iSCSI_Daemon:0 after complete monitor op (interval=30000)
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Target_iscsi2 after complete start op (interval=0)
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Target_iscsi2 after complete monitor op (interval=10000)
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Target_iscsi1 after complete monitor op (interval=0)
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_finalize_respond: join-8: Join complete.  Sending local LRM status to Cluster-Server-2
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: attrd_update_delegate: Sent update: (null)=(null) for localhost
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_NOT_DC: [ state=S_PENDING cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_LOG   
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: do_log: FSA: Input I_NOT_DC from do_cl_join_finalize_respond() received in state S_PENDING
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: notice: do_state_transition: State transition S_PENDING -> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_INTEGRATE_TIMER_STOP
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_FINALIZE_TIMER_STOP
Sep 10 15:25:14 Cluster-Server-1 attrd: [48713]: notice: attrd_local_callback: Sending full refresh (origin=crmd)
Sep 10 15:25:14 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: master-p_Device_drive:0 (10000)
Sep 10 15:25:14 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:0'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[3])
Sep 10 15:25:14 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-master-p_Device_drive.0" name="master-p_Device_drive:0" value="10000" />
Sep 10 15:25:14 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: pingd (100)
Sep 10 15:25:14 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
Sep 10 15:25:14 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:25:14 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:25:14 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:25:14 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:25:14 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 113 for master-p_Device_drive:0=10000 passed
Sep 10 15:25:14 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 115 for probe_complete=true passed
Sep 10 15:25:14 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 117 for pingd=100 passed
Sep 10 15:25:14 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:1'] does not exist
Sep 10 15:25:14 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update -22 for master-p_Device_drive:1=(null) passed
Sep 10 15:25:14 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:25:14 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:25:14 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:25:14 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: on_msg_add_rsc:client [48715] adds resource Target_iscsi3
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=14:14:7:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=Target_iscsi3_monitor_0
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc Target_iscsi3
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[28] on Target_iscsi3 for client 48715, its parameters: crm_feature_set=[3.0.6] iqn=[iqn.2005-07.com.example:vdisk.iscsi3] CRM_meta_timeout=[20000]  to the operation list.
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: info: rsc:Target_iscsi3 probe[28] (pid 55796)
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: on_msg_add_rsc:client [48715] adds resource Lun_iscsi3
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=15:14:7:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=Lun_iscsi3_monitor_0
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc Lun_iscsi3
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[29] on Lun_iscsi3 for client 48715, its parameters: path=[/dev/drive-CSD/iscsi3_iSCSI] crm_feature_set=[3.0.6] lun=[0] handler=[vdisk_blockio] device_name=[iscsi3] CRM_meta_timeout=[20000] target_iqn=[iqn.2005-07.com.example:vdisk.iscsi3]  to the operation list.
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: info: rsc:Lun_iscsi3 probe[29] (pid 55797)
SCSTTarget(Target_iscsi3)[55796]:	2012/09/10_15:25:14 DEBUG: Target_iscsi3 monitor : 7
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: WARN: Managed Target_iscsi3:monitor process 55796 exited with return code 7.
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: info: operation monitor[28] on Target_iscsi3 for client 48715: pid 55796 exited with return code 7
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce Target_iscsi3 after complete monitor op (interval=0)
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Target_iscsi3_monitor_0 (call=28, rc=7, cib-update=65, confirmed=true) not running
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'Target_iscsi3'
Sep 10 15:25:14 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 120 for pingd=100 passed
Sep 10 15:25:14 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 122 for probe_complete=true passed
SCSTLun(Lun_iscsi3)[55797]:	2012/09/10_15:25:14 INFO: Lun_iscsi3 monitor : 7
SCSTLun(Lun_iscsi3)[55797]:	2012/09/10_15:25:14 INFO: Lun_iscsi3 monitor : 7
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: WARN: Managed Lun_iscsi3:monitor process 55797 exited with return code 7.
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: info: operation monitor[29] on Lun_iscsi3 for client 48715: pid 55797 exited with return code 7
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce Lun_iscsi3 after complete monitor op (interval=0)
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Lun_iscsi3_monitor_0 (call=29, rc=7, cib-update=66, confirmed=true) not running
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'Lun_iscsi3'
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: attrd_update_delegate: Sent update: probe_complete=true for localhost
Sep 10 15:25:14 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crmd: probe_complete=true
Sep 10 15:25:14 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: true, Current: true, Stored: true
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=84:14:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=Target_iscsi3_start_0
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc Target_iscsi3
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation start[30] on Target_iscsi3 for client 48715, its parameters: crm_feature_set=[3.0.6] CRM_meta_name=[start] iqn=[iqn.2005-07.com.example:vdisk.iscsi3] CRM_meta_timeout=[240000]  to the operation list.
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: info: rsc:Target_iscsi3 start[30] (pid 55810)
SCSTTarget(Target_iscsi3)[55810]:	2012/09/10_15:25:14 INFO: target iqn.2005-07.com.example:vdisk.iscsi3: Starting...
SCSTTarget(Target_iscsi3)[55810]:	2012/09/10_15:25:14 INFO: target iqn.2005-07.com.example:vdisk.iscsi3: Starting...
SCSTTarget(Target_iscsi3)[55810]:	2012/09/10_15:25:14 INFO: Enabling target iqn.2005-07.com.example:vdisk.iscsi3
SCSTTarget(Target_iscsi3)[55810]:	2012/09/10_15:25:14 INFO: Enabling target iqn.2005-07.com.example:vdisk.iscsi3
SCSTTarget(Target_iscsi3)[55810]:	2012/09/10_15:25:14 DEBUG: SCST target iqn.2005-07.com.example:vdisk.iscsi3: Started.
SCSTTarget(Target_iscsi3)[55810]:	2012/09/10_15:25:14 DEBUG: Target_iscsi3 start : 0
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: info: Managed Target_iscsi3:start process 55810 exited with return code 0.
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: info: operation start[30] on Target_iscsi3 for client 48715: pid 55810 exited with return code 0
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce Target_iscsi3 after complete start op (interval=0)
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Target_iscsi3_start_0 (call=30, rc=0, cib-update=67, confirmed=true) ok
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending start op to history for 'Target_iscsi3'
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=85:14:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=Target_iscsi3_monitor_10000
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[31] on Target_iscsi3 for client 48715, its parameters: crm_feature_set=[3.0.6] CRM_meta_name=[monitor] iqn=[iqn.2005-07.com.example:vdisk.iscsi3] CRM_meta_timeout=[60000] CRM_meta_interval=[10000]  to the operation list.
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: info: rsc:Target_iscsi3 monitor[31] (pid 55830)
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=86:14:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=Lun_iscsi3_start_0
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc Lun_iscsi3
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation start[32] on Lun_iscsi3 for client 48715, its parameters: handler=[vdisk_blockio] CRM_meta_timeout=[60000] CRM_meta_name=[start] target_iqn=[iqn.2005-07.com.example:vdisk.iscsi3] path=[/dev/drive-CSD/iscsi3_iSCSI] crm_feature_set=[3.0.6] lun=[0] device_name=[iscsi3]  to the operation list.
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: info: rsc:Lun_iscsi3 start[32] (pid 55831)
SCSTLun(Lun_iscsi3)[55831]:	2012/09/10_15:25:14 INFO: Disabling target iqn.2005-07.com.example:vdisk.iscsi3
SCSTLun(Lun_iscsi3)[55831]:	2012/09/10_15:25:14 INFO: Disabling target iqn.2005-07.com.example:vdisk.iscsi3
SCSTLun(Lun_iscsi3)[55831]:	2012/09/10_15:25:14 INFO: Starting lun 0 on target iqn.2005-07.com.example:vdisk.iscsi3
SCSTTarget(Target_iscsi3)[55830]:	2012/09/10_15:25:14 DEBUG: Target_iscsi3 monitor : 0
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: info: Managed Target_iscsi3:monitor process 55830 exited with return code 0.
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: info: operation monitor[31] on Target_iscsi3 for client 48715: pid 55830 exited with return code 0
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce Target_iscsi3 after complete monitor op (interval=10000)
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Target_iscsi3_monitor_10000 (call=31, rc=0, cib-update=68, confirmed=false) ok
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'Target_iscsi3'
SCSTLun(Lun_iscsi3)[55831]:	2012/09/10_15:25:14 INFO: Starting lun 0 on target iqn.2005-07.com.example:vdisk.iscsi3
SCSTLun(Lun_iscsi3)[55831]:	2012/09/10_15:25:14 INFO: Opening device iscsi3, target iqn.2005-07.com.example:vdisk.iscsi3
SCSTLun(Lun_iscsi3)[55831]:	2012/09/10_15:25:14 INFO: Opening device iscsi3, target iqn.2005-07.com.example:vdisk.iscsi3
SCSTLun(Lun_iscsi3)[55831]:	2012/09/10_15:25:14 INFO: Adding LUN 0, device iscsi3, target iqn.2005-07.com.example:vdisk.iscsi3
SCSTLun(Lun_iscsi3)[55831]:	2012/09/10_15:25:14 INFO: Adding LUN 0, device iscsi3, target iqn.2005-07.com.example:vdisk.iscsi3
SCSTLun(Lun_iscsi3)[55831]:	2012/09/10_15:25:14 INFO: Enabling target iqn.2005-07.com.example:vdisk.iscsi3
SCSTLun(Lun_iscsi3)[55831]:	2012/09/10_15:25:14 INFO: Enabling target iqn.2005-07.com.example:vdisk.iscsi3
SCSTLun(Lun_iscsi3)[55831]:	2012/09/10_15:25:14 INFO: Started lun 0 on target iqn.2005-07.com.example:vdisk.iscsi3
SCSTLun(Lun_iscsi3)[55831]:	2012/09/10_15:25:14 INFO: Started lun 0 on target iqn.2005-07.com.example:vdisk.iscsi3
SCSTLun(Lun_iscsi3)[55831]:	2012/09/10_15:25:14 INFO: Lun_iscsi3 start : 0
SCSTLun(Lun_iscsi3)[55831]:	2012/09/10_15:25:14 INFO: Lun_iscsi3 start : 0
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: info: Managed Lun_iscsi3:start process 55831 exited with return code 0.
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: info: operation start[32] on Lun_iscsi3 for client 48715: pid 55831 exited with return code 0
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce Lun_iscsi3 after complete start op (interval=0)
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Lun_iscsi3_start_0 (call=32, rc=0, cib-update=69, confirmed=true) ok
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending start op to history for 'Lun_iscsi3'
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=87:14:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=Lun_iscsi3_monitor_10000
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[33] on Lun_iscsi3 for client 48715, its parameters: handler=[vdisk_blockio] CRM_meta_timeout=[20000] CRM_meta_name=[monitor] target_iqn=[iqn.2005-07.com.example:vdisk.iscsi3] path=[/dev/drive-CSD/iscsi3_iSCSI] crm_feature_set=[3.0.6] CRM_meta_interval=[10000] lun=[0] device_name=[iscsi3]  to the operation list.
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: info: rsc:Lun_iscsi3 monitor[33] (pid 55878)
SCSTLun(Lun_iscsi3)[55878]:	2012/09/10_15:25:14 INFO: Lun_iscsi3 monitor : 0
SCSTLun(Lun_iscsi3)[55878]:	2012/09/10_15:25:14 INFO: Lun_iscsi3 monitor : 0
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: info: Managed Lun_iscsi3:monitor process 55878 exited with return code 0.
Sep 10 15:25:14 Cluster-Server-1 lrmd: [48712]: info: operation monitor[33] on Lun_iscsi3 for client 48715: pid 55878 exited with return code 0
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce Lun_iscsi3 after complete monitor op (interval=10000)
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Lun_iscsi3_monitor_10000 (call=33, rc=0, cib-update=70, confirmed=false) ok
Sep 10 15:25:14 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'Lun_iscsi3'
Sep 10 15:25:15 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi2 monitor[21] (pid 55896)
Sep 10 15:25:15 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi2 monitor[23] (pid 55897)
SCSTTarget(Target_iscsi2)[55896]:	2012/09/10_15:25:15 DEBUG: Target_iscsi2 monitor : 0
Sep 10 15:25:15 Cluster-Server-1 lrmd: [48712]: info: operation monitor[21] on Target_iscsi2 for client 48715: pid 55896 exited with return code 0
SCSTLun(Lun_iscsi2)[55897]:	2012/09/10_15:25:15 INFO: Lun_iscsi2 monitor : 0
SCSTLun(Lun_iscsi2)[55897]:	2012/09/10_15:25:15 INFO: Lun_iscsi2 monitor : 0
Sep 10 15:25:15 Cluster-Server-1 lrmd: [48712]: info: operation monitor[23] on Lun_iscsi2 for client 48715: pid 55897 exited with return code 0
Sep 10 15:25:15 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_NFS_Server:0 monitor[8] (pid 55910)
Sep 10 15:25:15 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_iSCSI_Daemon:0 monitor[5] (pid 55911)
Sep 10 15:25:15 Cluster-Server-1 lrmd: [48712]: info: operation monitor[8] on p_NFS_Server:0 for client 48715: pid 55910 exited with return code 0
Sep 10 15:25:15 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_NFS_Server:0:monitor:stdout) nfsd running

Sep 10 15:25:15 Cluster-Server-1 lrmd: [48712]: info: operation monitor[5] on p_iSCSI_Daemon:0 for client 48715: pid 55911 exited with return code 0
Sep 10 15:25:18 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 55921)
drbd(p_Device_drive:0)[55921]:	2012/09/10_15:25:18 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:25:18 Cluster-Server-1 crm_attribute: [55951]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:25:18 Cluster-Server-1 crm_attribute: [55951]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:25:18 Cluster-Server-1 crm_attribute: [55951]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:25:18 Cluster-Server-1 crm_attribute: [55951]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:25:18 Cluster-Server-1 crm_attribute: [55951]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:25:18 Cluster-Server-1 crm_attribute: [55951]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:25:18 Cluster-Server-1 crm_attribute: [55951]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:25:18 Cluster-Server-1 crm_attribute: [55951]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:25:18 Cluster-Server-1 crm_attribute: [55951]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:25:18 Cluster-Server-1 crm_attribute: [55951]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:25:18 Cluster-Server-1 crm_attribute: [55951]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:25:18 Cluster-Server-1 crm_attribute: [55951]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:25:18 Cluster-Server-1 crm_attribute: [55951]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:25:18 Cluster-Server-1 crm_attribute: [55951]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:25:18 Cluster-Server-1 crm_attribute: [55951]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:25:18 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:25:18 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[55921]:	2012/09/10_15:25:18 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[55921]:	2012/09/10_15:25:18 DEBUG: drive: Command output: 
Sep 10 15:25:18 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:25:18 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 55921 exited with return code 8
Sep 10 15:25:22 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 56117)
Sep 10 15:25:24 Cluster-Server-1 attrd_updater: [56137]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:25:24 Cluster-Server-1 attrd_updater: [56137]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:25:24 Cluster-Server-1 attrd_updater: [56137]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:25:24 Cluster-Server-1 attrd_updater: [56137]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:25:24 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:25:24 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:25:24 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 56117 exited with return code 0
Sep 10 15:25:24 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi3 monitor[31] (pid 56138)
SCSTTarget(Target_iscsi3)[56138]:	2012/09/10_15:25:24 DEBUG: Target_iscsi3 monitor : 0
Sep 10 15:25:24 Cluster-Server-1 lrmd: [48712]: info: operation monitor[31] on Target_iscsi3 for client 48715: pid 56138 exited with return code 0
Sep 10 15:25:24 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi3 monitor[33] (pid 56144)
SCSTLun(Lun_iscsi3)[56144]:	2012/09/10_15:25:24 INFO: Lun_iscsi3 monitor : 0
SCSTLun(Lun_iscsi3)[56144]:	2012/09/10_15:25:24 INFO: Lun_iscsi3 monitor : 0
Sep 10 15:25:24 Cluster-Server-1 lrmd: [48712]: info: operation monitor[33] on Lun_iscsi3 for client 48715: pid 56144 exited with return code 0
Sep 10 15:25:25 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi2 monitor[21] (pid 56152)
Sep 10 15:25:25 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi2 monitor[23] (pid 56153)
SCSTTarget(Target_iscsi2)[56152]:	2012/09/10_15:25:25 DEBUG: Target_iscsi2 monitor : 0
Sep 10 15:25:25 Cluster-Server-1 lrmd: [48712]: info: operation monitor[21] on Target_iscsi2 for client 48715: pid 56152 exited with return code 0
SCSTLun(Lun_iscsi2)[56153]:	2012/09/10_15:25:25 INFO: Lun_iscsi2 monitor : 0
SCSTLun(Lun_iscsi2)[56153]:	2012/09/10_15:25:25 INFO: Lun_iscsi2 monitor : 0
Sep 10 15:25:25 Cluster-Server-1 lrmd: [48712]: info: operation monitor[23] on Lun_iscsi2 for client 48715: pid 56153 exited with return code 0
Sep 10 15:25:28 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 56166)
drbd(p_Device_drive:0)[56166]:	2012/09/10_15:25:28 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:25:28 Cluster-Server-1 crm_attribute: [56196]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:25:28 Cluster-Server-1 crm_attribute: [56196]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:25:28 Cluster-Server-1 crm_attribute: [56196]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:25:28 Cluster-Server-1 crm_attribute: [56196]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:25:28 Cluster-Server-1 crm_attribute: [56196]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:25:28 Cluster-Server-1 crm_attribute: [56196]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:25:28 Cluster-Server-1 crm_attribute: [56196]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:25:28 Cluster-Server-1 crm_attribute: [56196]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:25:28 Cluster-Server-1 crm_attribute: [56196]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:25:28 Cluster-Server-1 crm_attribute: [56196]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:25:28 Cluster-Server-1 crm_attribute: [56196]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:25:28 Cluster-Server-1 crm_attribute: [56196]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:25:28 Cluster-Server-1 crm_attribute: [56196]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:25:28 Cluster-Server-1 crm_attribute: [56196]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:25:28 Cluster-Server-1 crm_attribute: [56196]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:25:28 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:25:28 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[56166]:	2012/09/10_15:25:28 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[56166]:	2012/09/10_15:25:28 DEBUG: drive: Command output: 
Sep 10 15:25:28 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:25:28 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 56166 exited with return code 8
Sep 10 15:25:34 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 56203)
Sep 10 15:25:34 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi3 monitor[31] (pid 56219)
SCSTTarget(Target_iscsi3)[56219]:	2012/09/10_15:25:34 DEBUG: Target_iscsi3 monitor : 0
Sep 10 15:25:34 Cluster-Server-1 lrmd: [48712]: info: operation monitor[31] on Target_iscsi3 for client 48715: pid 56219 exited with return code 0
Sep 10 15:25:34 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi3 monitor[33] (pid 56225)
SCSTLun(Lun_iscsi3)[56225]:	2012/09/10_15:25:34 INFO: Lun_iscsi3 monitor : 0
SCSTLun(Lun_iscsi3)[56225]:	2012/09/10_15:25:34 INFO: Lun_iscsi3 monitor : 0
Sep 10 15:25:34 Cluster-Server-1 lrmd: [48712]: info: operation monitor[33] on Lun_iscsi3 for client 48715: pid 56225 exited with return code 0
Sep 10 15:25:35 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi2 monitor[21] (pid 56233)
Sep 10 15:25:35 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi2 monitor[23] (pid 56234)
SCSTTarget(Target_iscsi2)[56233]:	2012/09/10_15:25:35 DEBUG: Target_iscsi2 monitor : 0
Sep 10 15:25:35 Cluster-Server-1 lrmd: [48712]: info: operation monitor[21] on Target_iscsi2 for client 48715: pid 56233 exited with return code 0
SCSTLun(Lun_iscsi2)[56234]:	2012/09/10_15:25:35 INFO: Lun_iscsi2 monitor : 0
SCSTLun(Lun_iscsi2)[56234]:	2012/09/10_15:25:35 INFO: Lun_iscsi2 monitor : 0
Sep 10 15:25:35 Cluster-Server-1 lrmd: [48712]: info: operation monitor[23] on Lun_iscsi2 for client 48715: pid 56234 exited with return code 0
Sep 10 15:25:36 Cluster-Server-1 attrd_updater: [56249]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:25:36 Cluster-Server-1 attrd_updater: [56249]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:25:36 Cluster-Server-1 attrd_updater: [56249]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:25:36 Cluster-Server-1 attrd_updater: [56249]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:25:36 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:25:36 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:25:36 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 56203 exited with return code 0
Sep 10 15:25:38 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 56250)
drbd(p_Device_drive:0)[56250]:	2012/09/10_15:25:38 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:25:38 Cluster-Server-1 crm_attribute: [56280]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:25:38 Cluster-Server-1 crm_attribute: [56280]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:25:38 Cluster-Server-1 crm_attribute: [56280]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:25:38 Cluster-Server-1 crm_attribute: [56280]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:25:38 Cluster-Server-1 crm_attribute: [56280]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:25:38 Cluster-Server-1 crm_attribute: [56280]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:25:38 Cluster-Server-1 crm_attribute: [56280]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:25:38 Cluster-Server-1 crm_attribute: [56280]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:25:38 Cluster-Server-1 crm_attribute: [56280]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:25:38 Cluster-Server-1 crm_attribute: [56280]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:25:38 Cluster-Server-1 crm_attribute: [56280]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:25:38 Cluster-Server-1 crm_attribute: [56280]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:25:38 Cluster-Server-1 crm_attribute: [56280]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:25:38 Cluster-Server-1 crm_attribute: [56280]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:25:38 Cluster-Server-1 crm_attribute: [56280]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:25:38 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:25:38 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[56250]:	2012/09/10_15:25:38 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[56250]:	2012/09/10_15:25:38 DEBUG: drive: Command output: 
Sep 10 15:25:38 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:25:38 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 56250 exited with return code 8
Sep 10 15:25:41 Cluster-Server-1 cib: [48709]: info: crm_signal_dispatch: Invoking handler for signal 13: Broken pipe
Sep 10 15:25:41 Cluster-Server-1 cib: [48709]: info: cib_enable_writes: (Re)enabling disk writes
Sep 10 15:25:44 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi3 monitor[31] (pid 56289)
SCSTTarget(Target_iscsi3)[56289]:	2012/09/10_15:25:44 DEBUG: Target_iscsi3 monitor : 0
Sep 10 15:25:44 Cluster-Server-1 lrmd: [48712]: info: operation monitor[31] on Target_iscsi3 for client 48715: pid 56289 exited with return code 0
Sep 10 15:25:44 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi3 monitor[33] (pid 56295)
SCSTLun(Lun_iscsi3)[56295]:	2012/09/10_15:25:44 INFO: Lun_iscsi3 monitor : 0
SCSTLun(Lun_iscsi3)[56295]:	2012/09/10_15:25:44 INFO: Lun_iscsi3 monitor : 0
Sep 10 15:25:44 Cluster-Server-1 lrmd: [48712]: info: operation monitor[33] on Lun_iscsi3 for client 48715: pid 56295 exited with return code 0
Sep 10 15:25:45 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi2 monitor[21] (pid 56303)
Sep 10 15:25:45 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi2 monitor[23] (pid 56304)
Sep 10 15:25:45 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_NFS_Server:0 monitor[8] (pid 56305)
Sep 10 15:25:45 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_iSCSI_Daemon:0 monitor[5] (pid 56306)
Sep 10 15:25:45 Cluster-Server-1 lrmd: [48712]: info: operation monitor[8] on p_NFS_Server:0 for client 48715: pid 56305 exited with return code 0
Sep 10 15:25:45 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_NFS_Server:0:monitor:stdout) nfsd running

SCSTTarget(Target_iscsi2)[56303]:	2012/09/10_15:25:45 DEBUG: Target_iscsi2 monitor : 0
Sep 10 15:25:45 Cluster-Server-1 lrmd: [48712]: info: operation monitor[21] on Target_iscsi2 for client 48715: pid 56303 exited with return code 0
Sep 10 15:25:45 Cluster-Server-1 lrmd: [48712]: info: operation monitor[5] on p_iSCSI_Daemon:0 for client 48715: pid 56306 exited with return code 0
SCSTLun(Lun_iscsi2)[56304]:	2012/09/10_15:25:45 INFO: Lun_iscsi2 monitor : 0
SCSTLun(Lun_iscsi2)[56304]:	2012/09/10_15:25:45 INFO: Lun_iscsi2 monitor : 0
Sep 10 15:25:45 Cluster-Server-1 lrmd: [48712]: info: operation monitor[23] on Lun_iscsi2 for client 48715: pid 56304 exited with return code 0
Sep 10 15:25:46 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 56323)
Sep 10 15:25:48 Cluster-Server-1 attrd_updater: [56341]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:25:48 Cluster-Server-1 attrd_updater: [56341]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:25:48 Cluster-Server-1 attrd_updater: [56341]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:25:48 Cluster-Server-1 attrd_updater: [56341]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:25:48 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:25:48 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:25:48 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 56323 exited with return code 0
Sep 10 15:25:48 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 56392)
drbd(p_Device_drive:0)[56392]:	2012/09/10_15:25:48 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:25:48 Cluster-Server-1 crm_attribute: [56452]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:25:48 Cluster-Server-1 crm_attribute: [56452]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:25:48 Cluster-Server-1 crm_attribute: [56452]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:25:48 Cluster-Server-1 crm_attribute: [56452]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:25:48 Cluster-Server-1 crm_attribute: [56452]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:25:48 Cluster-Server-1 crm_attribute: [56452]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:25:48 Cluster-Server-1 crm_attribute: [56452]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:25:48 Cluster-Server-1 crm_attribute: [56452]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:25:48 Cluster-Server-1 crm_attribute: [56452]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:25:48 Cluster-Server-1 crm_attribute: [56452]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:25:48 Cluster-Server-1 crm_attribute: [56452]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:25:48 Cluster-Server-1 crm_attribute: [56452]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:25:48 Cluster-Server-1 crm_attribute: [56452]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:25:48 Cluster-Server-1 crm_attribute: [56452]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:25:48 Cluster-Server-1 crm_attribute: [56452]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:25:48 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:25:48 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[56392]:	2012/09/10_15:25:48 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[56392]:	2012/09/10_15:25:48 DEBUG: drive: Command output: 
Sep 10 15:25:48 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:25:48 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 56392 exited with return code 8
Sep 10 15:25:54 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi3 monitor[31] (pid 56615)
SCSTTarget(Target_iscsi3)[56615]:	2012/09/10_15:25:54 DEBUG: Target_iscsi3 monitor : 0
Sep 10 15:25:54 Cluster-Server-1 lrmd: [48712]: info: operation monitor[31] on Target_iscsi3 for client 48715: pid 56615 exited with return code 0
Sep 10 15:25:54 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi3 monitor[33] (pid 56621)
SCSTLun(Lun_iscsi3)[56621]:	2012/09/10_15:25:54 INFO: Lun_iscsi3 monitor : 0
SCSTLun(Lun_iscsi3)[56621]:	2012/09/10_15:25:54 INFO: Lun_iscsi3 monitor : 0
Sep 10 15:25:54 Cluster-Server-1 lrmd: [48712]: info: operation monitor[33] on Lun_iscsi3 for client 48715: pid 56621 exited with return code 0
Sep 10 15:25:55 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi2 monitor[21] (pid 56629)
Sep 10 15:25:55 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi2 monitor[23] (pid 56630)
SCSTTarget(Target_iscsi2)[56629]:	2012/09/10_15:25:55 DEBUG: Target_iscsi2 monitor : 0
Sep 10 15:25:55 Cluster-Server-1 lrmd: [48712]: info: operation monitor[21] on Target_iscsi2 for client 48715: pid 56629 exited with return code 0
SCSTLun(Lun_iscsi2)[56630]:	2012/09/10_15:25:55 INFO: Lun_iscsi2 monitor : 0
SCSTLun(Lun_iscsi2)[56630]:	2012/09/10_15:25:55 INFO: Lun_iscsi2 monitor : 0
Sep 10 15:25:55 Cluster-Server-1 lrmd: [48712]: info: operation monitor[23] on Lun_iscsi2 for client 48715: pid 56630 exited with return code 0
Sep 10 15:25:58 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 56653)
Sep 10 15:25:58 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 56669)
drbd(p_Device_drive:0)[56669]:	2012/09/10_15:25:58 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:25:58 Cluster-Server-1 crm_attribute: [56699]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:25:58 Cluster-Server-1 crm_attribute: [56699]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:25:58 Cluster-Server-1 crm_attribute: [56699]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:25:58 Cluster-Server-1 crm_attribute: [56699]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:25:58 Cluster-Server-1 crm_attribute: [56699]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:25:58 Cluster-Server-1 crm_attribute: [56699]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:25:58 Cluster-Server-1 crm_attribute: [56699]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:25:58 Cluster-Server-1 crm_attribute: [56699]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:25:58 Cluster-Server-1 crm_attribute: [56699]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:25:58 Cluster-Server-1 crm_attribute: [56699]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:25:58 Cluster-Server-1 crm_attribute: [56699]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:25:58 Cluster-Server-1 crm_attribute: [56699]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:25:58 Cluster-Server-1 crm_attribute: [56699]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:25:58 Cluster-Server-1 crm_attribute: [56699]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:25:58 Cluster-Server-1 crm_attribute: [56699]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:25:58 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:25:58 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[56669]:	2012/09/10_15:25:58 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[56669]:	2012/09/10_15:25:58 DEBUG: drive: Command output: 
Sep 10 15:25:58 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:25:58 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 56669 exited with return code 8
Sep 10 15:26:00 Cluster-Server-1 attrd_updater: [56708]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:26:00 Cluster-Server-1 attrd_updater: [56708]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:26:00 Cluster-Server-1 attrd_updater: [56708]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:26:00 Cluster-Server-1 attrd_updater: [56708]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:26:00 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:26:00 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:26:00 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 56653 exited with return code 0
Sep 10 15:26:04 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi3 monitor[31] (pid 56712)
SCSTTarget(Target_iscsi3)[56712]:	2012/09/10_15:26:04 DEBUG: Target_iscsi3 monitor : 0
Sep 10 15:26:04 Cluster-Server-1 lrmd: [48712]: info: operation monitor[31] on Target_iscsi3 for client 48715: pid 56712 exited with return code 0
Sep 10 15:26:04 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi3 monitor[33] (pid 56718)
SCSTLun(Lun_iscsi3)[56718]:	2012/09/10_15:26:04 INFO: Lun_iscsi3 monitor : 0
SCSTLun(Lun_iscsi3)[56718]:	2012/09/10_15:26:04 INFO: Lun_iscsi3 monitor : 0
Sep 10 15:26:04 Cluster-Server-1 lrmd: [48712]: info: operation monitor[33] on Lun_iscsi3 for client 48715: pid 56718 exited with return code 0
Sep 10 15:26:05 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi2 monitor[21] (pid 56726)
Sep 10 15:26:05 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi2 monitor[23] (pid 56727)
SCSTTarget(Target_iscsi2)[56726]:	2012/09/10_15:26:05 DEBUG: Target_iscsi2 monitor : 0
Sep 10 15:26:05 Cluster-Server-1 lrmd: [48712]: info: operation monitor[21] on Target_iscsi2 for client 48715: pid 56726 exited with return code 0
SCSTLun(Lun_iscsi2)[56727]:	2012/09/10_15:26:05 INFO: Lun_iscsi2 monitor : 0
SCSTLun(Lun_iscsi2)[56727]:	2012/09/10_15:26:05 INFO: Lun_iscsi2 monitor : 0
Sep 10 15:26:05 Cluster-Server-1 lrmd: [48712]: info: operation monitor[23] on Lun_iscsi2 for client 48715: pid 56727 exited with return code 0
Sep 10 15:26:08 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 56797)
drbd(p_Device_drive:0)[56797]:	2012/09/10_15:26:09 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:26:09 Cluster-Server-1 crm_attribute: [56827]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:26:09 Cluster-Server-1 crm_attribute: [56827]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:26:09 Cluster-Server-1 crm_attribute: [56827]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:26:09 Cluster-Server-1 crm_attribute: [56827]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:26:09 Cluster-Server-1 crm_attribute: [56827]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:26:09 Cluster-Server-1 crm_attribute: [56827]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:26:09 Cluster-Server-1 crm_attribute: [56827]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:26:09 Cluster-Server-1 crm_attribute: [56827]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:26:09 Cluster-Server-1 crm_attribute: [56827]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:26:09 Cluster-Server-1 crm_attribute: [56827]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:26:09 Cluster-Server-1 crm_attribute: [56827]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:26:09 Cluster-Server-1 crm_attribute: [56827]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:26:09 Cluster-Server-1 crm_attribute: [56827]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:26:09 Cluster-Server-1 crm_attribute: [56827]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:26:09 Cluster-Server-1 crm_attribute: [56827]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:26:09 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:26:09 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[56797]:	2012/09/10_15:26:09 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[56797]:	2012/09/10_15:26:09 DEBUG: drive: Command output: 
Sep 10 15:26:09 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:26:09 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 56797 exited with return code 8
Sep 10 15:26:10 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 56834)
Sep 10 15:26:12 Cluster-Server-1 attrd_updater: [56995]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:26:12 Cluster-Server-1 attrd_updater: [56995]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:26:12 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:26:12 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:26:12 Cluster-Server-1 attrd_updater: [56995]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:26:12 Cluster-Server-1 attrd_updater: [56995]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:26:12 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 56834 exited with return code 0
Sep 10 15:26:14 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi3 monitor[31] (pid 57084)
SCSTTarget(Target_iscsi3)[57084]:	2012/09/10_15:26:14 DEBUG: Target_iscsi3 monitor : 0
Sep 10 15:26:14 Cluster-Server-1 lrmd: [48712]: info: operation monitor[31] on Target_iscsi3 for client 48715: pid 57084 exited with return code 0
Sep 10 15:26:14 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi3 monitor[33] (pid 57090)
SCSTLun(Lun_iscsi3)[57090]:	2012/09/10_15:26:14 INFO: Lun_iscsi3 monitor : 0
SCSTLun(Lun_iscsi3)[57090]:	2012/09/10_15:26:14 INFO: Lun_iscsi3 monitor : 0
Sep 10 15:26:14 Cluster-Server-1 lrmd: [48712]: info: operation monitor[33] on Lun_iscsi3 for client 48715: pid 57090 exited with return code 0
Sep 10 15:26:15 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_NFS_Server:0 monitor[8] (pid 57103)
Sep 10 15:26:15 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_iSCSI_Daemon:0 monitor[5] (pid 57104)
Sep 10 15:26:15 Cluster-Server-1 lrmd: [48712]: info: operation monitor[8] on p_NFS_Server:0 for client 48715: pid 57103 exited with return code 0
Sep 10 15:26:15 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_NFS_Server:0:monitor:stdout) nfsd running

Sep 10 15:26:15 Cluster-Server-1 lrmd: [48712]: info: operation monitor[5] on p_iSCSI_Daemon:0 for client 48715: pid 57104 exited with return code 0
Sep 10 15:26:15 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi2 monitor[21] (pid 57109)
Sep 10 15:26:15 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi2 monitor[23] (pid 57110)
SCSTTarget(Target_iscsi2)[57109]:	2012/09/10_15:26:15 DEBUG: Target_iscsi2 monitor : 0
Sep 10 15:26:15 Cluster-Server-1 lrmd: [48712]: info: operation monitor[21] on Target_iscsi2 for client 48715: pid 57109 exited with return code 0
SCSTLun(Lun_iscsi2)[57110]:	2012/09/10_15:26:15 INFO: Lun_iscsi2 monitor : 0
SCSTLun(Lun_iscsi2)[57110]:	2012/09/10_15:26:15 INFO: Lun_iscsi2 monitor : 0
Sep 10 15:26:15 Cluster-Server-1 lrmd: [48712]: info: operation monitor[23] on Lun_iscsi2 for client 48715: pid 57110 exited with return code 0
Sep 10 15:26:19 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 57217)
drbd(p_Device_drive:0)[57217]:	2012/09/10_15:26:19 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:26:19 Cluster-Server-1 crm_attribute: [57247]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:26:19 Cluster-Server-1 crm_attribute: [57247]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:26:19 Cluster-Server-1 crm_attribute: [57247]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:26:19 Cluster-Server-1 crm_attribute: [57247]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:26:19 Cluster-Server-1 crm_attribute: [57247]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:26:19 Cluster-Server-1 crm_attribute: [57247]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:26:19 Cluster-Server-1 crm_attribute: [57247]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:26:19 Cluster-Server-1 crm_attribute: [57247]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:26:19 Cluster-Server-1 crm_attribute: [57247]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:26:19 Cluster-Server-1 crm_attribute: [57247]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:26:19 Cluster-Server-1 crm_attribute: [57247]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:26:19 Cluster-Server-1 crm_attribute: [57247]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:26:19 Cluster-Server-1 crm_attribute: [57247]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:26:19 Cluster-Server-1 crm_attribute: [57247]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:26:19 Cluster-Server-1 crm_attribute: [57247]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:26:19 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:26:19 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[57217]:	2012/09/10_15:26:19 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[57217]:	2012/09/10_15:26:19 DEBUG: drive: Command output: 
Sep 10 15:26:19 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:26:19 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 57217 exited with return code 8
Sep 10 15:26:22 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 57297)
Sep 10 15:26:24 Cluster-Server-1 attrd_updater: [57336]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:26:24 Cluster-Server-1 attrd_updater: [57336]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:26:24 Cluster-Server-1 attrd_updater: [57336]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:26:24 Cluster-Server-1 attrd_updater: [57336]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:26:24 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:26:24 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:26:24 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 57297 exited with return code 0
Sep 10 15:26:24 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi3 monitor[31] (pid 57348)
SCSTTarget(Target_iscsi3)[57348]:	2012/09/10_15:26:24 DEBUG: Target_iscsi3 monitor : 0
Sep 10 15:26:24 Cluster-Server-1 lrmd: [48712]: info: operation monitor[31] on Target_iscsi3 for client 48715: pid 57348 exited with return code 0
Sep 10 15:26:24 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi3 monitor[33] (pid 57354)
SCSTLun(Lun_iscsi3)[57354]:	2012/09/10_15:26:24 INFO: Lun_iscsi3 monitor : 0
SCSTLun(Lun_iscsi3)[57354]:	2012/09/10_15:26:24 INFO: Lun_iscsi3 monitor : 0
Sep 10 15:26:24 Cluster-Server-1 lrmd: [48712]: info: operation monitor[33] on Lun_iscsi3 for client 48715: pid 57354 exited with return code 0
Sep 10 15:26:25 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi2 monitor[21] (pid 57383)
Sep 10 15:26:25 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi2 monitor[23] (pid 57384)
SCSTTarget(Target_iscsi2)[57383]:	2012/09/10_15:26:25 DEBUG: Target_iscsi2 monitor : 0
SCSTLun(Lun_iscsi2)[57384]:	2012/09/10_15:26:25 INFO: Lun_iscsi2 monitor : 0
Sep 10 15:26:25 Cluster-Server-1 lrmd: [48712]: info: operation monitor[21] on Target_iscsi2 for client 48715: pid 57383 exited with return code 0
SCSTLun(Lun_iscsi2)[57384]:	2012/09/10_15:26:25 INFO: Lun_iscsi2 monitor : 0
Sep 10 15:26:25 Cluster-Server-1 lrmd: [48712]: info: operation monitor[23] on Lun_iscsi2 for client 48715: pid 57384 exited with return code 0
Sep 10 15:26:27 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [57459] registered
Sep 10 15:26:27 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:57459] disconnected.
Sep 10 15:26:27 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:57459] is unregistered
Sep 10 15:26:27 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [57461] registered
Sep 10 15:26:27 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:57461] disconnected.
Sep 10 15:26:27 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:57461] is unregistered
Sep 10 15:26:27 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [57463] registered
Sep 10 15:26:27 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:57463] disconnected.
Sep 10 15:26:27 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:57463] is unregistered
Sep 10 15:26:27 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [57465] registered
Sep 10 15:26:27 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:57465] disconnected.
Sep 10 15:26:27 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:57465] is unregistered
Sep 10 15:26:27 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [57474] registered
Sep 10 15:26:27 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:57474] disconnected.
Sep 10 15:26:27 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:57474] is unregistered
Sep 10 15:26:27 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [57483] registered
Sep 10 15:26:27 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:57483] disconnected.
Sep 10 15:26:27 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:57483] is unregistered
Sep 10 15:26:27 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [57490] registered
Sep 10 15:26:27 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:57490] disconnected.
Sep 10 15:26:27 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:57490] is unregistered
Sep 10 15:26:27 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [57497] registered
Sep 10 15:26:27 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:57497] disconnected.
Sep 10 15:26:27 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:57497] is unregistered
Sep 10 15:26:28 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [57504] registered
Sep 10 15:26:28 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:57504] disconnected.
Sep 10 15:26:28 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:57504] is unregistered
Sep 10 15:26:28 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [57512] registered
Sep 10 15:26:28 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:57512] disconnected.
Sep 10 15:26:28 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:57512] is unregistered
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_COUNT
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: crm_uptime: Current CPU usage is: 0s, 80000us
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: crm_compare_age: Loose: 80000 vs 390000 (usec)
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: info: do_election_count_vote: Election 11 (owner: Cluster-Server-2) lost: vote from Cluster-Server-2 (Uptime)
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: update_dc: Unset DC. Was Cluster-Server-2
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_CHECK
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: do_election_check: Ignore election check: we not in an election
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_PENDING: [ state=S_NOT_DC cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_LOG   
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: do_log: FSA: Input I_PENDING from do_election_count_vote() received in state S_NOT_DC
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: notice: do_state_transition: State transition S_NOT_DC -> S_PENDING [ input=I_PENDING cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_INTEGRATE_TIMER_STOP
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_FINALIZE_TIMER_STOP
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_START
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: crm_timer_start: Started Election Trigger (I_DC_TIMEOUT:20000ms), src=68
Sep 10 15:26:28 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for cib_apply_diff op
Sep 10 15:26:28 Cluster-Server-1 cib: [48709]: debug: Forking temp process write_cib_contents
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: handle_request: Raising I_JOIN_OFFER: join-9
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_JOIN_OFFER: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ]
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_CL_JOIN_REQUEST
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: info: update_dc: Set DC to Cluster-Server-2 (3.0.6)
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_offer_respond: do_cl_join_offer_respond added action A_DC_TIMER_STOP to the FSA
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:26:28 Cluster-Server-1 cib: [48709]: info: Managed write_cib_contents process 57523 exited with return code 0.
Sep 10 15:26:28 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:1'] does not exist
Sep 10 15:26:28 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update -22 for master-p_Device_drive:1=(null) passed
Sep 10 15:26:28 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:26:28 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:26:28 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:26:28 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: join_query_callback: Respond to join offer join-9
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: join_query_callback: Acknowledging Cluster-Server-2 as our DC
Sep 10 15:26:28 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 125 for pingd=100 passed
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: handle_request: Raising I_JOIN_RESULT: join-9
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_JOIN_RESULT: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ]
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_CL_JOIN_RESULT
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_finalize_respond: Confirming join join-9: join_ack_nack
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_NFS_Server:0 after complete start op (interval=0)
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_NFS_Server:0 after complete monitor op (interval=30000)
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce LVM_drive after complete monitor op (interval=0)
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Lun_iscsi1 after complete monitor op (interval=0)
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_PingD:0 after complete monitor op (interval=0)
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_PingD:0 after complete monitor op (interval=10000)
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Lun_iscsi2 after complete start op (interval=0)
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Lun_iscsi2 after complete monitor op (interval=10000)
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_Device_drive:0 after complete monitor op (interval=0)
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_Device_drive:0 after complete monitor op (interval=10000)
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_iSCSI_Daemon:0 after complete monitor op (interval=0)
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_iSCSI_Daemon:0 after complete monitor op (interval=30000)
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Target_iscsi2 after complete start op (interval=0)
Sep 10 15:26:28 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 127 for probe_complete=true passed
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Target_iscsi2 after complete monitor op (interval=10000)
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Target_iscsi3 after complete start op (interval=0)
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Target_iscsi3 after complete monitor op (interval=10000)
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Target_iscsi1 after complete monitor op (interval=0)
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Lun_iscsi3 after complete start op (interval=0)
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Lun_iscsi3 after complete monitor op (interval=10000)
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_finalize_respond: join-9: Join complete.  Sending local LRM status to Cluster-Server-2
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: attrd_update_delegate: Sent update: (null)=(null) for localhost
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_NOT_DC: [ state=S_PENDING cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_LOG   
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: do_log: FSA: Input I_NOT_DC from do_cl_join_finalize_respond() received in state S_PENDING
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: notice: do_state_transition: State transition S_PENDING -> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_INTEGRATE_TIMER_STOP
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_FINALIZE_TIMER_STOP
Sep 10 15:26:28 Cluster-Server-1 attrd: [48713]: notice: attrd_local_callback: Sending full refresh (origin=crmd)
Sep 10 15:26:28 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: master-p_Device_drive:0 (10000)
Sep 10 15:26:28 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for cib_replace op
Sep 10 15:26:28 Cluster-Server-1 lrmd: [48712]: debug: on_msg_add_rsc:client [48715] adds resource FS_nfs1
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=16:15:7:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=FS_nfs1_monitor_0
Sep 10 15:26:28 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc FS_nfs1
Sep 10 15:26:28 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[34] on FS_nfs1 for client 48715, its parameters: crm_feature_set=[3.0.6] device=[/dev/drive-CSD/nfs1_NFS] directory=[/volumes/nfs1] force_clones=[false] fstype=[xfs] CRM_meta_timeout=[20000]  to the operation list.
Sep 10 15:26:28 Cluster-Server-1 lrmd: [48712]: info: rsc:FS_nfs1 probe[34] (pid 57536)
Sep 10 15:26:28 Cluster-Server-1 lrmd: [48712]: debug: on_msg_add_rsc:client [48715] adds resource ExportFS_nfs1
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=17:15:7:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=ExportFS_nfs1_monitor_0
Sep 10 15:26:28 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc ExportFS_nfs1
Sep 10 15:26:28 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[35] on ExportFS_nfs1 for client 48715, its parameters: crm_feature_set=[3.0.6] options=[rw,insecure,async,no_subtree_check,root_squash,no_all_squash,anonuid=1directory=[/volumes/nfs1] fsid=[1955f364-fb4b-11e1-b02e-000c290247c7] clientspec=[*] CRM_meta_timeout=[20000]  to the operation list.
Sep 10 15:26:28 Cluster-Server-1 lrmd: [48712]: info: rsc:ExportFS_nfs1 probe[35] (pid 57538)
Sep 10 15:26:28 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:0'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[3])
Sep 10 15:26:28 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-master-p_Device_drive.0" name="master-p_Device_drive:0" value="10000" />
Sep 10 15:26:28 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: pingd (100)
Sep 10 15:26:28 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
Sep 10 15:26:28 Cluster-Server-1 cib: [48709]: debug: Forking temp process write_cib_contents
Sep 10 15:26:28 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:26:28 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:26:28 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:1'] does not exist
Sep 10 15:26:28 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update -22 for master-p_Device_drive:1=(null) passed
Sep 10 15:26:28 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:26:28 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:26:28 Cluster-Server-1 cib: [48709]: info: Managed write_cib_contents process 57539 exited with return code 0.
Sep 10 15:26:28 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:26:28 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:26:28 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:26:28 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:26:28 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 129 for master-p_Device_drive:0=10000 passed
Sep 10 15:26:28 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 131 for probe_complete=true passed
exportfs(ExportFS_nfs1)[57538]:	2012/09/10_15:26:28 INFO: Directory /volumes/nfs1 is not exported to * (stopped).
exportfs(ExportFS_nfs1)[57538]:	2012/09/10_15:26:28 INFO: Directory /volumes/nfs1 is not exported to * (stopped).
Sep 10 15:26:28 Cluster-Server-1 lrmd: [48712]: WARN: Managed ExportFS_nfs1:monitor process 57538 exited with return code 7.
Sep 10 15:26:28 Cluster-Server-1 lrmd: [48712]: info: operation monitor[35] on ExportFS_nfs1 for client 48715: pid 57538 exited with return code 7
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce ExportFS_nfs1 after complete monitor op (interval=0)
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation ExportFS_nfs1_monitor_0 (call=35, rc=7, cib-update=74, confirmed=true) not running
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'ExportFS_nfs1'
Sep 10 15:26:28 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 134 for pingd=100 passed
Sep 10 15:26:28 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 136 for probe_complete=true passed
Sep 10 15:26:28 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 138 for pingd=100 passed
Sep 10 15:26:28 Cluster-Server-1 lrmd: [48712]: WARN: Managed FS_nfs1:monitor process 57536 exited with return code 7.
Sep 10 15:26:28 Cluster-Server-1 lrmd: [48712]: info: operation monitor[34] on FS_nfs1 for client 48715: pid 57536 exited with return code 7
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce FS_nfs1 after complete monitor op (interval=0)
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation FS_nfs1_monitor_0 (call=34, rc=7, cib-update=75, confirmed=true) not running
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'FS_nfs1'
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: attrd_update_delegate: Sent update: probe_complete=true for localhost
Sep 10 15:26:28 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crmd: probe_complete=true
Sep 10 15:26:28 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: true, Current: true, Stored: true
Sep 10 15:26:28 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=90:16:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=FS_nfs1_start_0
Sep 10 15:26:28 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc FS_nfs1
Sep 10 15:26:28 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation start[36] on FS_nfs1 for client 48715, its parameters: crm_feature_set=[3.0.6] device=[/dev/drive-CSD/nfs1_NFS] directory=[/volumes/nfs1] CRM_meta_name=[start] force_clones=[false] CRM_meta_timeout=[60000] fstype=[xfs]  to the operation list.
Sep 10 15:26:28 Cluster-Server-1 lrmd: [48712]: info: rsc:FS_nfs1 start[36] (pid 57591)
Filesystem(FS_nfs1)[57591]:	2012/09/10_15:26:28 INFO: Running start for /dev/drive-CSD/nfs1_NFS on /volumes/nfs1
Filesystem(FS_nfs1)[57591]:	2012/09/10_15:26:28 INFO: Running start for /dev/drive-CSD/nfs1_NFS on /volumes/nfs1
Sep 10 15:26:29 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 57648)
drbd(p_Device_drive:0)[57648]:	2012/09/10_15:26:29 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:26:29 Cluster-Server-1 crm_attribute: [57678]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:26:29 Cluster-Server-1 crm_attribute: [57678]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:26:29 Cluster-Server-1 crm_attribute: [57678]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:26:29 Cluster-Server-1 crm_attribute: [57678]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:26:29 Cluster-Server-1 crm_attribute: [57678]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:26:29 Cluster-Server-1 crm_attribute: [57678]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:26:29 Cluster-Server-1 crm_attribute: [57678]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:26:29 Cluster-Server-1 crm_attribute: [57678]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:26:29 Cluster-Server-1 crm_attribute: [57678]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:26:29 Cluster-Server-1 crm_attribute: [57678]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:26:29 Cluster-Server-1 crm_attribute: [57678]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:26:29 Cluster-Server-1 crm_attribute: [57678]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:26:29 Cluster-Server-1 crm_attribute: [57678]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:26:29 Cluster-Server-1 crm_attribute: [57678]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:26:29 Cluster-Server-1 crm_attribute: [57678]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:26:29 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:26:29 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[57648]:	2012/09/10_15:26:29 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[57648]:	2012/09/10_15:26:29 DEBUG: drive: Command output: 
Sep 10 15:26:29 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:26:29 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 57648 exited with return code 8
Sep 10 15:26:30 Cluster-Server-1 lrmd: [48712]: info: Managed FS_nfs1:start process 57591 exited with return code 0.
Sep 10 15:26:30 Cluster-Server-1 lrmd: [48712]: info: operation start[36] on FS_nfs1 for client 48715: pid 57591 exited with return code 0
Sep 10 15:26:30 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce FS_nfs1 after complete start op (interval=0)
Sep 10 15:26:30 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation FS_nfs1_start_0 (call=36, rc=0, cib-update=76, confirmed=true) ok
Sep 10 15:26:30 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending start op to history for 'FS_nfs1'
Sep 10 15:26:30 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=91:16:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=FS_nfs1_monitor_20000
Sep 10 15:26:30 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[37] on FS_nfs1 for client 48715, its parameters: fstype=[xfs] CRM_meta_timeout=[40000] CRM_meta_name=[monitor] crm_feature_set=[3.0.6] device=[/dev/drive-CSD/nfs1_NFS] force_clones=[false] CRM_meta_interval=[20000] directory=[/volumes/nfs1]  to the operation list.
Sep 10 15:26:30 Cluster-Server-1 lrmd: [48712]: info: rsc:FS_nfs1 monitor[37] (pid 57706)
Sep 10 15:26:30 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=92:16:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=ExportFS_nfs1_start_0
Sep 10 15:26:30 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc ExportFS_nfs1
Sep 10 15:26:30 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation start[38] on ExportFS_nfs1 for client 48715, its parameters: crm_feature_set=[3.0.6] directory=[/volumes/nfs1] fsid=[1955f364-fb4b-11e1-b02e-000c290247c7] CRM_meta_name=[start] options=[rw,insecure,async,no_subtree_check,root_squash,no_all_squash,anonuid=1clientspec=[*] CRM_meta_timeout=[40000]  to the operation list.
Sep 10 15:26:30 Cluster-Server-1 lrmd: [48712]: info: rsc:ExportFS_nfs1 start[38] (pid 57707)
exportfs(ExportFS_nfs1)[57707]:	2012/09/10_15:26:30 INFO: Directory /volumes/nfs1 is not exported to * (stopped).
exportfs(ExportFS_nfs1)[57707]:	2012/09/10_15:26:30 INFO: Directory /volumes/nfs1 is not exported to * (stopped).
exportfs(ExportFS_nfs1)[57707]:	2012/09/10_15:26:30 INFO: Exporting file system ...
exportfs(ExportFS_nfs1)[57707]:	2012/09/10_15:26:30 INFO: Exporting file system ...
Sep 10 15:26:30 Cluster-Server-1 lrmd: [48712]: info: Managed FS_nfs1:monitor process 57706 exited with return code 0.
Sep 10 15:26:30 Cluster-Server-1 lrmd: [48712]: info: operation monitor[37] on FS_nfs1 for client 48715: pid 57706 exited with return code 0
Sep 10 15:26:30 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce FS_nfs1 after complete monitor op (interval=20000)
Sep 10 15:26:30 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation FS_nfs1_monitor_20000 (call=37, rc=0, cib-update=77, confirmed=false) ok
Sep 10 15:26:30 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'FS_nfs1'
exportfs(ExportFS_nfs1)[57707]:	2012/09/10_15:26:30 INFO: exporting *:/volumes/nfs1
exportfs(ExportFS_nfs1)[57707]:	2012/09/10_15:26:30 INFO: exporting *:/volumes/nfs1
exportfs(ExportFS_nfs1)[57707]:	2012/09/10_15:26:30 WARNING: rmtab backup /volumes/nfs1/.rmtab not found or not readable.
exportfs(ExportFS_nfs1)[57707]:	2012/09/10_15:26:30 WARNING: rmtab backup /volumes/nfs1/.rmtab not found or not readable.
exportfs(ExportFS_nfs1)[57707]:	2012/09/10_15:26:30 INFO: File system exported
exportfs(ExportFS_nfs1)[57707]:	2012/09/10_15:26:30 INFO: File system exported
Sep 10 15:26:30 Cluster-Server-1 lrmd: [48712]: info: Managed ExportFS_nfs1:start process 57707 exited with return code 0.
Sep 10 15:26:30 Cluster-Server-1 lrmd: [48712]: info: operation start[38] on ExportFS_nfs1 for client 48715: pid 57707 exited with return code 0
Sep 10 15:26:30 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce ExportFS_nfs1 after complete start op (interval=0)
Sep 10 15:26:30 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation ExportFS_nfs1_start_0 (call=38, rc=0, cib-update=78, confirmed=true) ok
Sep 10 15:26:30 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending start op to history for 'ExportFS_nfs1'
Sep 10 15:26:30 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=93:16:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=ExportFS_nfs1_monitor_10000
Sep 10 15:26:30 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[39] on ExportFS_nfs1 for client 48715, its parameters: options=[rw,insecure,async,no_subtree_check,root_squash,no_all_squash,anonuid=1CRM_meta_timeout=[20000] CRM_meta_name=[monitor] fsid=[1955f364-fb4b-11e1-b02e-000c290247c7] crm_feature_set=[3.0.6] CRM_meta_interval=[10000] clientspec=[*] directory=[/volumes/nfs1]  to the operation list.
Sep 10 15:26:30 Cluster-Server-1 lrmd: [48712]: info: rsc:ExportFS_nfs1 monitor[39] (pid 57773)
exportfs(ExportFS_nfs1)[57773]:	2012/09/10_15:26:30 INFO: Directory /volumes/nfs1 is exported to * (started).
exportfs(ExportFS_nfs1)[57773]:	2012/09/10_15:26:30 INFO: Directory /volumes/nfs1 is exported to * (started).
Sep 10 15:26:30 Cluster-Server-1 lrmd: [48712]: info: Managed ExportFS_nfs1:monitor process 57773 exited with return code 0.
Sep 10 15:26:30 Cluster-Server-1 lrmd: [48712]: info: operation monitor[39] on ExportFS_nfs1 for client 48715: pid 57773 exited with return code 0
Sep 10 15:26:30 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce ExportFS_nfs1 after complete monitor op (interval=10000)
Sep 10 15:26:30 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation ExportFS_nfs1_monitor_10000 (call=39, rc=0, cib-update=79, confirmed=false) ok
Sep 10 15:26:30 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'ExportFS_nfs1'
Sep 10 15:26:34 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 57801)
Sep 10 15:26:34 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi3 monitor[31] (pid 57817)
SCSTTarget(Target_iscsi3)[57817]:	2012/09/10_15:26:34 DEBUG: Target_iscsi3 monitor : 0
Sep 10 15:26:34 Cluster-Server-1 lrmd: [48712]: info: operation monitor[31] on Target_iscsi3 for client 48715: pid 57817 exited with return code 0
Sep 10 15:26:34 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi3 monitor[33] (pid 57823)
SCSTLun(Lun_iscsi3)[57823]:	2012/09/10_15:26:34 INFO: Lun_iscsi3 monitor : 0
SCSTLun(Lun_iscsi3)[57823]:	2012/09/10_15:26:34 INFO: Lun_iscsi3 monitor : 0
Sep 10 15:26:34 Cluster-Server-1 lrmd: [48712]: info: operation monitor[33] on Lun_iscsi3 for client 48715: pid 57823 exited with return code 0
Sep 10 15:26:35 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi2 monitor[21] (pid 57831)
Sep 10 15:26:35 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi2 monitor[23] (pid 57832)
SCSTTarget(Target_iscsi2)[57831]:	2012/09/10_15:26:35 DEBUG: Target_iscsi2 monitor : 0
Sep 10 15:26:35 Cluster-Server-1 lrmd: [48712]: info: operation monitor[21] on Target_iscsi2 for client 48715: pid 57831 exited with return code 0
SCSTLun(Lun_iscsi2)[57832]:	2012/09/10_15:26:35 INFO: Lun_iscsi2 monitor : 0
SCSTLun(Lun_iscsi2)[57832]:	2012/09/10_15:26:35 INFO: Lun_iscsi2 monitor : 0
Sep 10 15:26:35 Cluster-Server-1 lrmd: [48712]: info: operation monitor[23] on Lun_iscsi2 for client 48715: pid 57832 exited with return code 0
Sep 10 15:26:36 Cluster-Server-1 attrd_updater: [57849]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:26:36 Cluster-Server-1 attrd_updater: [57849]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:26:36 Cluster-Server-1 attrd_updater: [57849]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:26:36 Cluster-Server-1 attrd_updater: [57849]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:26:36 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:26:36 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:26:36 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 57801 exited with return code 0
Sep 10 15:26:39 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 58046)
drbd(p_Device_drive:0)[58046]:	2012/09/10_15:26:39 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:26:39 Cluster-Server-1 crm_attribute: [58077]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:26:39 Cluster-Server-1 crm_attribute: [58077]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:26:39 Cluster-Server-1 crm_attribute: [58077]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:26:39 Cluster-Server-1 crm_attribute: [58077]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:26:39 Cluster-Server-1 crm_attribute: [58077]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:26:39 Cluster-Server-1 crm_attribute: [58077]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:26:39 Cluster-Server-1 crm_attribute: [58077]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:26:39 Cluster-Server-1 crm_attribute: [58077]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:26:39 Cluster-Server-1 crm_attribute: [58077]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:26:39 Cluster-Server-1 crm_attribute: [58077]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:26:39 Cluster-Server-1 crm_attribute: [58077]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:26:39 Cluster-Server-1 crm_attribute: [58077]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:26:39 Cluster-Server-1 crm_attribute: [58077]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:26:39 Cluster-Server-1 crm_attribute: [58077]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:26:39 Cluster-Server-1 crm_attribute: [58077]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:26:39 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:26:39 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[58046]:	2012/09/10_15:26:39 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[58046]:	2012/09/10_15:26:39 DEBUG: drive: Command output: 
Sep 10 15:26:39 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:26:39 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 58046 exited with return code 8
Sep 10 15:26:40 Cluster-Server-1 lrmd: [48712]: debug: rsc:ExportFS_nfs1 monitor[39] (pid 58350)
exportfs(ExportFS_nfs1)[58350]:	2012/09/10_15:26:41 INFO: Directory /volumes/nfs1 is exported to * (started).
exportfs(ExportFS_nfs1)[58350]:	2012/09/10_15:26:41 INFO: Directory /volumes/nfs1 is exported to * (started).
Sep 10 15:26:41 Cluster-Server-1 lrmd: [48712]: info: operation monitor[39] on ExportFS_nfs1 for client 48715: pid 58350 exited with return code 0
Sep 10 15:26:44 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi3 monitor[31] (pid 58368)
Sep 10 15:26:44 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi3 monitor[33] (pid 58371)
SCSTTarget(Target_iscsi3)[58368]:	2012/09/10_15:26:44 DEBUG: Target_iscsi3 monitor : 0
Sep 10 15:26:44 Cluster-Server-1 lrmd: [48712]: info: operation monitor[31] on Target_iscsi3 for client 48715: pid 58368 exited with return code 0
SCSTLun(Lun_iscsi3)[58371]:	2012/09/10_15:26:44 INFO: Lun_iscsi3 monitor : 0
SCSTLun(Lun_iscsi3)[58371]:	2012/09/10_15:26:44 INFO: Lun_iscsi3 monitor : 0
Sep 10 15:26:44 Cluster-Server-1 lrmd: [48712]: info: operation monitor[33] on Lun_iscsi3 for client 48715: pid 58371 exited with return code 0
Sep 10 15:26:45 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_NFS_Server:0 monitor[8] (pid 58382)
Sep 10 15:26:45 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_iSCSI_Daemon:0 monitor[5] (pid 58383)
Sep 10 15:26:45 Cluster-Server-1 lrmd: [48712]: info: operation monitor[8] on p_NFS_Server:0 for client 48715: pid 58382 exited with return code 0
Sep 10 15:26:45 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_NFS_Server:0:monitor:stdout) nfsd running

Sep 10 15:26:45 Cluster-Server-1 lrmd: [48712]: info: operation monitor[5] on p_iSCSI_Daemon:0 for client 48715: pid 58383 exited with return code 0
Sep 10 15:26:45 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi2 monitor[21] (pid 58388)
Sep 10 15:26:45 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi2 monitor[23] (pid 58389)
SCSTTarget(Target_iscsi2)[58388]:	2012/09/10_15:26:45 DEBUG: Target_iscsi2 monitor : 0
Sep 10 15:26:45 Cluster-Server-1 lrmd: [48712]: info: operation monitor[21] on Target_iscsi2 for client 48715: pid 58388 exited with return code 0
SCSTLun(Lun_iscsi2)[58389]:	2012/09/10_15:26:45 INFO: Lun_iscsi2 monitor : 0
SCSTLun(Lun_iscsi2)[58389]:	2012/09/10_15:26:45 INFO: Lun_iscsi2 monitor : 0
Sep 10 15:26:45 Cluster-Server-1 lrmd: [48712]: info: operation monitor[23] on Lun_iscsi2 for client 48715: pid 58389 exited with return code 0
Sep 10 15:26:46 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 58402)
Sep 10 15:26:48 Cluster-Server-1 attrd_updater: [58420]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:26:48 Cluster-Server-1 attrd_updater: [58420]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:26:48 Cluster-Server-1 attrd_updater: [58420]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:26:48 Cluster-Server-1 attrd_updater: [58420]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:26:48 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:26:48 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:26:48 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 58402 exited with return code 0
Sep 10 15:26:49 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 58421)
drbd(p_Device_drive:0)[58421]:	2012/09/10_15:26:49 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:26:49 Cluster-Server-1 crm_attribute: [58451]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:26:49 Cluster-Server-1 crm_attribute: [58451]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:26:49 Cluster-Server-1 crm_attribute: [58451]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:26:49 Cluster-Server-1 crm_attribute: [58451]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:26:49 Cluster-Server-1 crm_attribute: [58451]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:26:49 Cluster-Server-1 crm_attribute: [58451]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:26:49 Cluster-Server-1 crm_attribute: [58451]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:26:49 Cluster-Server-1 crm_attribute: [58451]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:26:49 Cluster-Server-1 crm_attribute: [58451]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:26:49 Cluster-Server-1 crm_attribute: [58451]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:26:49 Cluster-Server-1 crm_attribute: [58451]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:26:49 Cluster-Server-1 crm_attribute: [58451]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:26:49 Cluster-Server-1 crm_attribute: [58451]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:26:49 Cluster-Server-1 crm_attribute: [58451]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:26:49 Cluster-Server-1 crm_attribute: [58451]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:26:49 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:26:49 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[58421]:	2012/09/10_15:26:49 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[58421]:	2012/09/10_15:26:49 DEBUG: drive: Command output: 
Sep 10 15:26:49 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:26:49 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 58421 exited with return code 8
Sep 10 15:26:50 Cluster-Server-1 lrmd: [48712]: debug: rsc:FS_nfs1 monitor[37] (pid 58720)
Sep 10 15:26:50 Cluster-Server-1 lrmd: [48712]: info: operation monitor[37] on FS_nfs1 for client 48715: pid 58720 exited with return code 0
Sep 10 15:26:51 Cluster-Server-1 lrmd: [48712]: debug: rsc:ExportFS_nfs1 monitor[39] (pid 58752)
exportfs(ExportFS_nfs1)[58752]:	2012/09/10_15:26:51 INFO: Directory /volumes/nfs1 is exported to * (started).
exportfs(ExportFS_nfs1)[58752]:	2012/09/10_15:26:51 INFO: Directory /volumes/nfs1 is exported to * (started).
Sep 10 15:26:51 Cluster-Server-1 lrmd: [48712]: info: operation monitor[39] on ExportFS_nfs1 for client 48715: pid 58752 exited with return code 0
Sep 10 15:26:54 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi3 monitor[31] (pid 58836)
Sep 10 15:26:54 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi3 monitor[33] (pid 58837)
SCSTTarget(Target_iscsi3)[58836]:	2012/09/10_15:26:54 DEBUG: Target_iscsi3 monitor : 0
Sep 10 15:26:54 Cluster-Server-1 lrmd: [48712]: info: operation monitor[31] on Target_iscsi3 for client 48715: pid 58836 exited with return code 0
SCSTLun(Lun_iscsi3)[58837]:	2012/09/10_15:26:54 INFO: Lun_iscsi3 monitor : 0
SCSTLun(Lun_iscsi3)[58837]:	2012/09/10_15:26:54 INFO: Lun_iscsi3 monitor : 0
Sep 10 15:26:54 Cluster-Server-1 lrmd: [48712]: info: operation monitor[33] on Lun_iscsi3 for client 48715: pid 58837 exited with return code 0
Sep 10 15:26:55 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi2 monitor[21] (pid 58861)
Sep 10 15:26:55 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi2 monitor[23] (pid 58862)
SCSTTarget(Target_iscsi2)[58861]:	2012/09/10_15:26:55 DEBUG: Target_iscsi2 monitor : 0
Sep 10 15:26:55 Cluster-Server-1 lrmd: [48712]: info: operation monitor[21] on Target_iscsi2 for client 48715: pid 58861 exited with return code 0
SCSTLun(Lun_iscsi2)[58862]:	2012/09/10_15:26:55 INFO: Lun_iscsi2 monitor : 0
SCSTLun(Lun_iscsi2)[58862]:	2012/09/10_15:26:55 INFO: Lun_iscsi2 monitor : 0
Sep 10 15:26:55 Cluster-Server-1 lrmd: [48712]: info: operation monitor[23] on Lun_iscsi2 for client 48715: pid 58862 exited with return code 0
Sep 10 15:26:58 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 58907)
Sep 10 15:26:59 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 58944)
drbd(p_Device_drive:0)[58944]:	2012/09/10_15:26:59 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:26:59 Cluster-Server-1 crm_attribute: [58974]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:26:59 Cluster-Server-1 crm_attribute: [58974]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:26:59 Cluster-Server-1 crm_attribute: [58974]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:26:59 Cluster-Server-1 crm_attribute: [58974]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:26:59 Cluster-Server-1 crm_attribute: [58974]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:26:59 Cluster-Server-1 crm_attribute: [58974]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:26:59 Cluster-Server-1 crm_attribute: [58974]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:26:59 Cluster-Server-1 crm_attribute: [58974]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:26:59 Cluster-Server-1 crm_attribute: [58974]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:26:59 Cluster-Server-1 crm_attribute: [58974]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:26:59 Cluster-Server-1 crm_attribute: [58974]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:26:59 Cluster-Server-1 crm_attribute: [58974]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:26:59 Cluster-Server-1 crm_attribute: [58974]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:26:59 Cluster-Server-1 crm_attribute: [58974]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:26:59 Cluster-Server-1 crm_attribute: [58974]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:26:59 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:26:59 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[58944]:	2012/09/10_15:26:59 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[58944]:	2012/09/10_15:26:59 DEBUG: drive: Command output: 
Sep 10 15:26:59 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:26:59 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 58944 exited with return code 8
Sep 10 15:27:00 Cluster-Server-1 attrd_updater: [58994]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:27:00 Cluster-Server-1 attrd_updater: [58994]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:27:00 Cluster-Server-1 attrd_updater: [58994]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:27:00 Cluster-Server-1 attrd_updater: [58994]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:27:00 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:27:00 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:27:00 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 58907 exited with return code 0
Sep 10 15:27:01 Cluster-Server-1 lrmd: [48712]: debug: rsc:ExportFS_nfs1 monitor[39] (pid 59016)
exportfs(ExportFS_nfs1)[59016]:	2012/09/10_15:27:01 INFO: Directory /volumes/nfs1 is exported to * (started).
exportfs(ExportFS_nfs1)[59016]:	2012/09/10_15:27:01 INFO: Directory /volumes/nfs1 is exported to * (started).
Sep 10 15:27:01 Cluster-Server-1 lrmd: [48712]: info: operation monitor[39] on ExportFS_nfs1 for client 48715: pid 59016 exited with return code 0
Sep 10 15:27:04 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi3 monitor[31] (pid 59082)
Sep 10 15:27:04 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi3 monitor[33] (pid 59088)
SCSTTarget(Target_iscsi3)[59082]:	2012/09/10_15:27:04 DEBUG: Target_iscsi3 monitor : 0
Sep 10 15:27:04 Cluster-Server-1 lrmd: [48712]: info: operation monitor[31] on Target_iscsi3 for client 48715: pid 59082 exited with return code 0
SCSTLun(Lun_iscsi3)[59088]:	2012/09/10_15:27:04 INFO: Lun_iscsi3 monitor : 0
SCSTLun(Lun_iscsi3)[59088]:	2012/09/10_15:27:04 INFO: Lun_iscsi3 monitor : 0
Sep 10 15:27:04 Cluster-Server-1 lrmd: [48712]: info: operation monitor[33] on Lun_iscsi3 for client 48715: pid 59088 exited with return code 0
Sep 10 15:27:05 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi2 monitor[21] (pid 59119)
Sep 10 15:27:05 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi2 monitor[23] (pid 59120)
SCSTTarget(Target_iscsi2)[59119]:	2012/09/10_15:27:05 DEBUG: Target_iscsi2 monitor : 0
Sep 10 15:27:05 Cluster-Server-1 lrmd: [48712]: info: operation monitor[21] on Target_iscsi2 for client 48715: pid 59119 exited with return code 0
SCSTLun(Lun_iscsi2)[59120]:	2012/09/10_15:27:05 INFO: Lun_iscsi2 monitor : 0
SCSTLun(Lun_iscsi2)[59120]:	2012/09/10_15:27:05 INFO: Lun_iscsi2 monitor : 0
Sep 10 15:27:05 Cluster-Server-1 lrmd: [48712]: info: operation monitor[23] on Lun_iscsi2 for client 48715: pid 59120 exited with return code 0
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [59163] registered
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:59163] disconnected.
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:59163] is unregistered
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [59165] registered
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:59165] disconnected.
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:59165] is unregistered
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [59167] registered
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:59167] disconnected.
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:59167] is unregistered
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [59169] registered
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:59169] disconnected.
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:59169] is unregistered
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [59178] registered
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:59178] disconnected.
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:59178] is unregistered
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [59187] registered
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:59187] disconnected.
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:59187] is unregistered
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [59194] registered
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:59194] disconnected.
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:59194] is unregistered
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [59201] registered
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:59201] disconnected.
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:59201] is unregistered
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [59208] registered
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:59208] disconnected.
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:59208] is unregistered
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [59216] registered
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:59216] disconnected.
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:59216] is unregistered
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_COUNT
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: crm_uptime: Current CPU usage is: 0s, 80000us
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: crm_compare_age: Loose: 80000 vs 430000 (usec)
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: info: do_election_count_vote: Election 12 (owner: Cluster-Server-2) lost: vote from Cluster-Server-2 (Uptime)
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: update_dc: Unset DC. Was Cluster-Server-2
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_CHECK
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: do_election_check: Ignore election check: we not in an election
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_PENDING: [ state=S_NOT_DC cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_LOG   
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: do_log: FSA: Input I_PENDING from do_election_count_vote() received in state S_NOT_DC
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: notice: do_state_transition: State transition S_NOT_DC -> S_PENDING [ input=I_PENDING cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_INTEGRATE_TIMER_STOP
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_FINALIZE_TIMER_STOP
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_START
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: crm_timer_start: Started Election Trigger (I_DC_TIMEOUT:20000ms), src=76
Sep 10 15:27:06 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for cib_apply_diff op
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: handle_request: Raising I_JOIN_OFFER: join-10
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_JOIN_OFFER: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ]
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_CL_JOIN_REQUEST
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: info: update_dc: Set DC to Cluster-Server-2 (3.0.6)
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_offer_respond: do_cl_join_offer_respond added action A_DC_TIMER_STOP to the FSA
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:27:06 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:1'] does not exist
Sep 10 15:27:06 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update -22 for master-p_Device_drive:1=(null) passed
Sep 10 15:27:06 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:27:06 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:27:06 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:27:06 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:27:06 Cluster-Server-1 cib: [48709]: debug: Forking temp process write_cib_contents
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: join_query_callback: Respond to join offer join-10
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: join_query_callback: Acknowledging Cluster-Server-2 as our DC
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: handle_request: Raising I_JOIN_RESULT: join-10
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_JOIN_RESULT: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ]
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_CL_JOIN_RESULT
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_finalize_respond: Confirming join join-10: join_ack_nack
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce ExportFS_nfs1 after complete start op (interval=0)
Sep 10 15:27:06 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 141 for pingd=100 passed
Sep 10 15:27:06 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 143 for probe_complete=true passed
Sep 10 15:27:06 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for cib_replace op
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce ExportFS_nfs1 after complete monitor op (interval=10000)
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_NFS_Server:0 after complete start op (interval=0)
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_NFS_Server:0 after complete monitor op (interval=30000)
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce FS_nfs1 after complete start op (interval=0)
Sep 10 15:27:06 Cluster-Server-1 cib: [48709]: info: Managed write_cib_contents process 59238 exited with return code 0.
Sep 10 15:27:06 Cluster-Server-1 cib: [48709]: debug: Forking temp process write_cib_contents
Sep 10 15:27:06 Cluster-Server-1 cib: [48709]: debug: xmlfromIPC: Peer disconnected
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce FS_nfs1 after complete monitor op (interval=20000)
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce LVM_drive after complete monitor op (interval=0)
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Lun_iscsi1 after complete monitor op (interval=0)
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_PingD:0 after complete monitor op (interval=0)
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_PingD:0 after complete monitor op (interval=10000)
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Lun_iscsi2 after complete start op (interval=0)
Sep 10 15:27:06 Cluster-Server-1 cib: [48709]: info: Managed write_cib_contents process 59250 exited with return code 0.
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Lun_iscsi2 after complete monitor op (interval=10000)
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_Device_drive:0 after complete monitor op (interval=0)
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_Device_drive:0 after complete monitor op (interval=10000)
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_iSCSI_Daemon:0 after complete monitor op (interval=0)
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_iSCSI_Daemon:0 after complete monitor op (interval=30000)
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Target_iscsi2 after complete start op (interval=0)
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Target_iscsi2 after complete monitor op (interval=10000)
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Target_iscsi3 after complete start op (interval=0)
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Target_iscsi3 after complete monitor op (interval=10000)
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Target_iscsi1 after complete monitor op (interval=0)
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Lun_iscsi3 after complete start op (interval=0)
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Lun_iscsi3 after complete monitor op (interval=10000)
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_finalize_respond: join-10: Join complete.  Sending local LRM status to Cluster-Server-2
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: attrd_update_delegate: Sent update: (null)=(null) for localhost
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_NOT_DC: [ state=S_PENDING cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_LOG   
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: do_log: FSA: Input I_NOT_DC from do_cl_join_finalize_respond() received in state S_PENDING
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: notice: do_state_transition: State transition S_PENDING -> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_INTEGRATE_TIMER_STOP
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_FINALIZE_TIMER_STOP
Sep 10 15:27:06 Cluster-Server-1 attrd: [48713]: notice: attrd_local_callback: Sending full refresh (origin=crmd)
Sep 10 15:27:06 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: master-p_Device_drive:0 (10000)
Sep 10 15:27:06 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:0'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[3])
Sep 10 15:27:06 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-master-p_Device_drive.0" name="master-p_Device_drive:0" value="10000" />
Sep 10 15:27:06 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: pingd (100)
Sep 10 15:27:06 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
Sep 10 15:27:06 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:27:06 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:27:06 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:27:06 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:27:06 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 145 for master-p_Device_drive:0=10000 passed
Sep 10 15:27:06 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 147 for probe_complete=true passed
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: on_msg_add_rsc:client [48715] adds resource FS_nfs2
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=18:17:7:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=FS_nfs2_monitor_0
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc FS_nfs2
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[40] on FS_nfs2 for client 48715, its parameters: crm_feature_set=[3.0.6] device=[/dev/drive-CSD/nfs2_NFS] directory=[/volumes/nfs2] force_clones=[false] fstype=[xfs] CRM_meta_timeout=[20000]  to the operation list.
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: info: rsc:FS_nfs2 probe[40] (pid 59265)
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: on_msg_add_rsc:client [48715] adds resource ExportFS_nfs2
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=19:17:7:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=ExportFS_nfs2_monitor_0
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc ExportFS_nfs2
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[41] on ExportFS_nfs2 for client 48715, its parameters: crm_feature_set=[3.0.6] options=[rw,insecure,async,no_subtree_check,root_squash,no_all_squash,anonuid=1directory=[/volumes/nfs2] fsid=[2fbaecfe-fb4b-11e1-a319-000c290247c7] clientspec=[*] CRM_meta_timeout=[20000]  to the operation list.
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: info: rsc:ExportFS_nfs2 probe[41] (pid 59266)
exportfs(ExportFS_nfs2)[59266]:	2012/09/10_15:27:06 INFO: Directory /volumes/nfs2 is not exported to * (stopped).
exportfs(ExportFS_nfs2)[59266]:	2012/09/10_15:27:06 INFO: Directory /volumes/nfs2 is not exported to * (stopped).
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: WARN: Managed ExportFS_nfs2:monitor process 59266 exited with return code 7.
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: info: operation monitor[41] on ExportFS_nfs2 for client 48715: pid 59266 exited with return code 7
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce ExportFS_nfs2 after complete monitor op (interval=0)
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation ExportFS_nfs2_monitor_0 (call=41, rc=7, cib-update=83, confirmed=true) not running
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'ExportFS_nfs2'
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: WARN: Managed FS_nfs2:monitor process 59265 exited with return code 7.
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: info: operation monitor[40] on FS_nfs2 for client 48715: pid 59265 exited with return code 7
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce FS_nfs2 after complete monitor op (interval=0)
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation FS_nfs2_monitor_0 (call=40, rc=7, cib-update=84, confirmed=true) not running
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'FS_nfs2'
Sep 10 15:27:06 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:1'] does not exist
Sep 10 15:27:06 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update -22 for master-p_Device_drive:1=(null) passed
Sep 10 15:27:06 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:27:06 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:27:06 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:27:06 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:27:06 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 149 for pingd=100 passed
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: attrd_update_delegate: Sent update: probe_complete=true for localhost
Sep 10 15:27:06 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crmd: probe_complete=true
Sep 10 15:27:06 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: true, Current: true, Stored: true
Sep 10 15:27:06 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 152 for pingd=100 passed
Sep 10 15:27:06 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 154 for probe_complete=true passed
Sep 10 15:27:06 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=100:18:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=FS_nfs2_start_0
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc FS_nfs2
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation start[42] on FS_nfs2 for client 48715, its parameters: crm_feature_set=[3.0.6] device=[/dev/drive-CSD/nfs2_NFS] directory=[/volumes/nfs2] CRM_meta_name=[start] force_clones=[false] CRM_meta_timeout=[60000] fstype=[xfs]  to the operation list.
Sep 10 15:27:06 Cluster-Server-1 lrmd: [48712]: info: rsc:FS_nfs2 start[42] (pid 59309)
Filesystem(FS_nfs2)[59309]:	2012/09/10_15:27:06 INFO: Running start for /dev/drive-CSD/nfs2_NFS on /volumes/nfs2
Filesystem(FS_nfs2)[59309]:	2012/09/10_15:27:06 INFO: Running start for /dev/drive-CSD/nfs2_NFS on /volumes/nfs2
Sep 10 15:27:09 Cluster-Server-1 lrmd: [48712]: info: Managed FS_nfs2:start process 59309 exited with return code 0.
Sep 10 15:27:09 Cluster-Server-1 lrmd: [48712]: info: operation start[42] on FS_nfs2 for client 48715: pid 59309 exited with return code 0
Sep 10 15:27:09 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce FS_nfs2 after complete start op (interval=0)
Sep 10 15:27:09 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation FS_nfs2_start_0 (call=42, rc=0, cib-update=85, confirmed=true) ok
Sep 10 15:27:09 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending start op to history for 'FS_nfs2'
Sep 10 15:27:09 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=101:18:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=FS_nfs2_monitor_20000
Sep 10 15:27:09 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[43] on FS_nfs2 for client 48715, its parameters: fstype=[xfs] CRM_meta_timeout=[40000] CRM_meta_name=[monitor] crm_feature_set=[3.0.6] device=[/dev/drive-CSD/nfs2_NFS] force_clones=[false] CRM_meta_interval=[20000] directory=[/volumes/nfs2]  to the operation list.
Sep 10 15:27:09 Cluster-Server-1 lrmd: [48712]: info: rsc:FS_nfs2 monitor[43] (pid 59389)
Sep 10 15:27:09 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=102:18:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=ExportFS_nfs2_start_0
Sep 10 15:27:09 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc ExportFS_nfs2
Sep 10 15:27:09 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation start[44] on ExportFS_nfs2 for client 48715, its parameters: crm_feature_set=[3.0.6] directory=[/volumes/nfs2] fsid=[2fbaecfe-fb4b-11e1-a319-000c290247c7] CRM_meta_name=[start] options=[rw,insecure,async,no_subtree_check,root_squash,no_all_squash,anonuid=1clientspec=[*] CRM_meta_timeout=[40000]  to the operation list.
Sep 10 15:27:09 Cluster-Server-1 lrmd: [48712]: info: rsc:ExportFS_nfs2 start[44] (pid 59390)
exportfs(ExportFS_nfs2)[59390]:	2012/09/10_15:27:09 INFO: Directory /volumes/nfs2 is not exported to * (stopped).
exportfs(ExportFS_nfs2)[59390]:	2012/09/10_15:27:09 INFO: Directory /volumes/nfs2 is not exported to * (stopped).
exportfs(ExportFS_nfs2)[59390]:	2012/09/10_15:27:09 INFO: Exporting file system ...
exportfs(ExportFS_nfs2)[59390]:	2012/09/10_15:27:09 INFO: Exporting file system ...
Sep 10 15:27:09 Cluster-Server-1 lrmd: [48712]: info: Managed FS_nfs2:monitor process 59389 exited with return code 0.
Sep 10 15:27:09 Cluster-Server-1 lrmd: [48712]: info: operation monitor[43] on FS_nfs2 for client 48715: pid 59389 exited with return code 0
Sep 10 15:27:09 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce FS_nfs2 after complete monitor op (interval=20000)
Sep 10 15:27:09 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation FS_nfs2_monitor_20000 (call=43, rc=0, cib-update=86, confirmed=false) ok
Sep 10 15:27:09 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'FS_nfs2'
exportfs(ExportFS_nfs2)[59390]:	2012/09/10_15:27:09 INFO: exporting *:/volumes/nfs2
exportfs(ExportFS_nfs2)[59390]:	2012/09/10_15:27:09 INFO: exporting *:/volumes/nfs2
exportfs(ExportFS_nfs2)[59390]:	2012/09/10_15:27:09 WARNING: rmtab backup /volumes/nfs2/.rmtab not found or not readable.
exportfs(ExportFS_nfs2)[59390]:	2012/09/10_15:27:09 WARNING: rmtab backup /volumes/nfs2/.rmtab not found or not readable.
exportfs(ExportFS_nfs2)[59390]:	2012/09/10_15:27:09 INFO: File system exported
exportfs(ExportFS_nfs2)[59390]:	2012/09/10_15:27:09 INFO: File system exported
Sep 10 15:27:09 Cluster-Server-1 lrmd: [48712]: info: Managed ExportFS_nfs2:start process 59390 exited with return code 0.
Sep 10 15:27:09 Cluster-Server-1 lrmd: [48712]: info: operation start[44] on ExportFS_nfs2 for client 48715: pid 59390 exited with return code 0
Sep 10 15:27:09 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce ExportFS_nfs2 after complete start op (interval=0)
Sep 10 15:27:09 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation ExportFS_nfs2_start_0 (call=44, rc=0, cib-update=87, confirmed=true) ok
Sep 10 15:27:09 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending start op to history for 'ExportFS_nfs2'
Sep 10 15:27:09 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=103:18:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=ExportFS_nfs2_monitor_10000
Sep 10 15:27:09 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[45] on ExportFS_nfs2 for client 48715, its parameters: options=[rw,insecure,async,no_subtree_check,root_squash,no_all_squash,anonuid=1CRM_meta_timeout=[20000] CRM_meta_name=[monitor] fsid=[2fbaecfe-fb4b-11e1-a319-000c290247c7] crm_feature_set=[3.0.6] CRM_meta_interval=[10000] clientspec=[*] directory=[/volumes/nfs2]  to the operation list.
Sep 10 15:27:09 Cluster-Server-1 lrmd: [48712]: info: rsc:ExportFS_nfs2 monitor[45] (pid 59456)
exportfs(ExportFS_nfs2)[59456]:	2012/09/10_15:27:09 INFO: Directory /volumes/nfs2 is exported to * (started).
exportfs(ExportFS_nfs2)[59456]:	2012/09/10_15:27:09 INFO: Directory /volumes/nfs2 is exported to * (started).
Sep 10 15:27:09 Cluster-Server-1 lrmd: [48712]: info: Managed ExportFS_nfs2:monitor process 59456 exited with return code 0.
Sep 10 15:27:09 Cluster-Server-1 lrmd: [48712]: info: operation monitor[45] on ExportFS_nfs2 for client 48715: pid 59456 exited with return code 0
Sep 10 15:27:09 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce ExportFS_nfs2 after complete monitor op (interval=10000)
Sep 10 15:27:09 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation ExportFS_nfs2_monitor_10000 (call=45, rc=0, cib-update=88, confirmed=false) ok
Sep 10 15:27:09 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'ExportFS_nfs2'
Sep 10 15:27:09 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 59467)
drbd(p_Device_drive:0)[59467]:	2012/09/10_15:27:09 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:27:09 Cluster-Server-1 crm_attribute: [59497]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:27:09 Cluster-Server-1 crm_attribute: [59497]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:27:09 Cluster-Server-1 crm_attribute: [59497]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:27:09 Cluster-Server-1 crm_attribute: [59497]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:27:09 Cluster-Server-1 crm_attribute: [59497]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:27:09 Cluster-Server-1 crm_attribute: [59497]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:27:09 Cluster-Server-1 crm_attribute: [59497]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:27:09 Cluster-Server-1 crm_attribute: [59497]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:27:09 Cluster-Server-1 crm_attribute: [59497]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:27:09 Cluster-Server-1 crm_attribute: [59497]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:27:09 Cluster-Server-1 crm_attribute: [59497]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:27:09 Cluster-Server-1 crm_attribute: [59497]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:27:09 Cluster-Server-1 crm_attribute: [59497]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:27:09 Cluster-Server-1 crm_attribute: [59497]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:27:09 Cluster-Server-1 crm_attribute: [59497]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:27:09 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:27:09 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[59467]:	2012/09/10_15:27:09 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[59467]:	2012/09/10_15:27:09 DEBUG: drive: Command output: 
Sep 10 15:27:09 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:27:09 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 59467 exited with return code 8
Sep 10 15:27:10 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 59514)
Sep 10 15:27:11 Cluster-Server-1 lrmd: [48712]: debug: rsc:FS_nfs1 monitor[37] (pid 59530)
Sep 10 15:27:11 Cluster-Server-1 lrmd: [48712]: debug: rsc:ExportFS_nfs1 monitor[39] (pid 59557)
Sep 10 15:27:11 Cluster-Server-1 lrmd: [48712]: info: operation monitor[37] on FS_nfs1 for client 48715: pid 59530 exited with return code 0
exportfs(ExportFS_nfs1)[59557]:	2012/09/10_15:27:11 INFO: Directory /volumes/nfs1 is exported to * (started).
exportfs(ExportFS_nfs1)[59557]:	2012/09/10_15:27:11 INFO: Directory /volumes/nfs1 is exported to * (started).
Sep 10 15:27:11 Cluster-Server-1 lrmd: [48712]: info: operation monitor[39] on ExportFS_nfs1 for client 48715: pid 59557 exited with return code 0
Sep 10 15:27:12 Cluster-Server-1 attrd_updater: [59634]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:27:12 Cluster-Server-1 attrd_updater: [59634]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:27:12 Cluster-Server-1 attrd_updater: [59634]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:27:12 Cluster-Server-1 attrd_updater: [59634]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:27:12 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:27:12 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:27:12 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 59514 exited with return code 0
Sep 10 15:27:14 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi3 monitor[31] (pid 59642)
Sep 10 15:27:14 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi3 monitor[33] (pid 59645)
SCSTTarget(Target_iscsi3)[59642]:	2012/09/10_15:27:14 DEBUG: Target_iscsi3 monitor : 0
Sep 10 15:27:14 Cluster-Server-1 lrmd: [48712]: info: operation monitor[31] on Target_iscsi3 for client 48715: pid 59642 exited with return code 0
SCSTLun(Lun_iscsi3)[59645]:	2012/09/10_15:27:14 INFO: Lun_iscsi3 monitor : 0
SCSTLun(Lun_iscsi3)[59645]:	2012/09/10_15:27:14 INFO: Lun_iscsi3 monitor : 0
Sep 10 15:27:14 Cluster-Server-1 lrmd: [48712]: info: operation monitor[33] on Lun_iscsi3 for client 48715: pid 59645 exited with return code 0
Sep 10 15:27:15 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_NFS_Server:0 monitor[8] (pid 59799)
Sep 10 15:27:15 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_iSCSI_Daemon:0 monitor[5] (pid 59800)
Sep 10 15:27:15 Cluster-Server-1 lrmd: [48712]: info: operation monitor[8] on p_NFS_Server:0 for client 48715: pid 59799 exited with return code 0
Sep 10 15:27:15 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_NFS_Server:0:monitor:stdout) nfsd running

Sep 10 15:27:15 Cluster-Server-1 lrmd: [48712]: info: operation monitor[5] on p_iSCSI_Daemon:0 for client 48715: pid 59800 exited with return code 0
Sep 10 15:27:15 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi2 monitor[21] (pid 59844)
Sep 10 15:27:15 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi2 monitor[23] (pid 59849)
SCSTTarget(Target_iscsi2)[59844]:	2012/09/10_15:27:15 DEBUG: Target_iscsi2 monitor : 0
Sep 10 15:27:15 Cluster-Server-1 lrmd: [48712]: info: operation monitor[21] on Target_iscsi2 for client 48715: pid 59844 exited with return code 0
SCSTLun(Lun_iscsi2)[59849]:	2012/09/10_15:27:15 INFO: Lun_iscsi2 monitor : 0
SCSTLun(Lun_iscsi2)[59849]:	2012/09/10_15:27:15 INFO: Lun_iscsi2 monitor : 0
Sep 10 15:27:15 Cluster-Server-1 lrmd: [48712]: info: operation monitor[23] on Lun_iscsi2 for client 48715: pid 59849 exited with return code 0
Sep 10 15:27:19 Cluster-Server-1 lrmd: [48712]: debug: rsc:ExportFS_nfs2 monitor[45] (pid 60147)
exportfs(ExportFS_nfs2)[60147]:	2012/09/10_15:27:19 INFO: Directory /volumes/nfs2 is exported to * (started).
exportfs(ExportFS_nfs2)[60147]:	2012/09/10_15:27:19 INFO: Directory /volumes/nfs2 is exported to * (started).
Sep 10 15:27:19 Cluster-Server-1 lrmd: [48712]: info: operation monitor[45] on ExportFS_nfs2 for client 48715: pid 60147 exited with return code 0
Sep 10 15:27:19 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 60206)
drbd(p_Device_drive:0)[60206]:	2012/09/10_15:27:19 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:27:19 Cluster-Server-1 crm_attribute: [60245]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:27:19 Cluster-Server-1 crm_attribute: [60245]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:27:19 Cluster-Server-1 crm_attribute: [60245]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:27:19 Cluster-Server-1 crm_attribute: [60245]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:27:19 Cluster-Server-1 crm_attribute: [60245]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:27:19 Cluster-Server-1 crm_attribute: [60245]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:27:19 Cluster-Server-1 crm_attribute: [60245]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:27:19 Cluster-Server-1 crm_attribute: [60245]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:27:19 Cluster-Server-1 crm_attribute: [60245]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:27:19 Cluster-Server-1 crm_attribute: [60245]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:27:19 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:27:19 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
Sep 10 15:27:19 Cluster-Server-1 crm_attribute: [60245]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:27:19 Cluster-Server-1 crm_attribute: [60245]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:27:19 Cluster-Server-1 crm_attribute: [60245]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:27:19 Cluster-Server-1 crm_attribute: [60245]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:27:19 Cluster-Server-1 crm_attribute: [60245]: info: crm_xml_cleanup: Cleaning up memory from libxml2
drbd(p_Device_drive:0)[60206]:	2012/09/10_15:27:19 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[60206]:	2012/09/10_15:27:19 DEBUG: drive: Command output: 
Sep 10 15:27:19 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 60206 exited with return code 8
Sep 10 15:27:19 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:27:21 Cluster-Server-1 lrmd: [48712]: debug: rsc:ExportFS_nfs1 monitor[39] (pid 60298)
exportfs(ExportFS_nfs1)[60298]:	2012/09/10_15:27:21 INFO: Directory /volumes/nfs1 is exported to * (started).
exportfs(ExportFS_nfs1)[60298]:	2012/09/10_15:27:21 INFO: Directory /volumes/nfs1 is exported to * (started).
Sep 10 15:27:21 Cluster-Server-1 lrmd: [48712]: info: operation monitor[39] on ExportFS_nfs1 for client 48715: pid 60298 exited with return code 0
Sep 10 15:27:22 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 60314)
Sep 10 15:27:23 Cluster-Server-1 cib: [48709]: info: apply_xml_diff: Digest mis-match: expected 3a2ae85c18bcc0caac2c445e79aaabc7, calculated 2fa5d48bc5c3cbbc1f57b6aeca085db0
Sep 10 15:27:23 Cluster-Server-1 cib: [48709]: notice: cib_process_diff: Diff 0.15.39 -> 0.16.1 not applied to 0.15.39: Failed application of an update diff
Sep 10 15:27:23 Cluster-Server-1 cib: [48709]: info: cib_server_process_diff: Requesting re-sync from peer
Sep 10 15:27:23 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for cib_replace op
Sep 10 15:27:23 Cluster-Server-1 cib: [48709]: info: cib_replace_notify: Replaced: -1.-1.-1 -> 0.16.1 from Cluster-Server-2
Sep 10 15:27:23 Cluster-Server-1 crmd: [48715]: debug: do_cib_replaced: Updating the CIB after a replace: DC=false
Sep 10 15:27:23 Cluster-Server-1 attrd: [48713]: info: do_cib_replaced: Sending full refresh
Sep 10 15:27:23 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: master-p_Device_drive:0 (10000)
Sep 10 15:27:23 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:0'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[3])
Sep 10 15:27:23 Cluster-Server-1 cib: [48709]: debug: Forking temp process write_cib_contents
Sep 10 15:27:23 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-master-p_Device_drive.0" name="master-p_Device_drive:0" value="10000" />
Sep 10 15:27:23 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: pingd (100)
Sep 10 15:27:23 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
Sep 10 15:27:23 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:27:23 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:27:23 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:27:23 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:27:23 Cluster-Server-1 crmd: [48715]: debug: cancel_op: Cancelling op 39 for ExportFS_nfs1 (ExportFS_nfs1:39)
Sep 10 15:27:23 Cluster-Server-1 lrmd: [48712]: info: cancel_op: operation monitor[39] on ExportFS_nfs1 for client 48715, its parameters: options=[rw,insecure,async,no_subtree_check,root_squash,no_all_squash,anonuid=1CRM_meta_timeout=[20000] CRM_meta_name=[monitor] fsid=[1955f364-fb4b-11e1-b02e-000c290247c7] crm_feature_set=[3.0.6] CRM_meta_interval=[10000] clientspec=[*] directory=[/volumes/nfs1]  cancelled
Sep 10 15:27:23 Cluster-Server-1 lrmd: [48712]: debug: on_msg_cancel_op: operation 39 cancelled
Sep 10 15:27:23 Cluster-Server-1 crmd: [48715]: debug: cancel_op: Op 39 for ExportFS_nfs1 (ExportFS_nfs1:39): cancelled
Sep 10 15:27:23 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=95:19:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=ExportFS_nfs1_stop_0
Sep 10 15:27:23 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation stop[46] on ExportFS_nfs1 for client 48715, its parameters: crm_feature_set=[3.0.6] CRM_meta_name=[stop] CRM_meta_timeout=[10000]  to the operation list.
Sep 10 15:27:23 Cluster-Server-1 lrmd: [48712]: info: rsc:ExportFS_nfs1 stop[46] (pid 60393)
Sep 10 15:27:23 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation ExportFS_nfs1_monitor_10000 (call=39, status=1, cib-update=0, confirmed=true) Cancelled
Sep 10 15:27:23 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'ExportFS_nfs1'
exportfs(ExportFS_nfs1)[60393]:	2012/09/10_15:27:23 INFO: Directory /volumes/nfs1 is exported to * (started).
exportfs(ExportFS_nfs1)[60393]:	2012/09/10_15:27:23 INFO: Directory /volumes/nfs1 is exported to * (started).
exportfs(ExportFS_nfs1)[60393]:	2012/09/10_15:27:23 INFO: Un-exporting file system ...
Sep 10 15:27:23 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 156 for master-p_Device_drive:0=10000 passed
exportfs(ExportFS_nfs1)[60393]:	2012/09/10_15:27:23 INFO: Un-exporting file system ...
Sep 10 15:27:23 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 158 for probe_complete=true passed
exportfs(ExportFS_nfs1)[60393]:	2012/09/10_15:27:23 INFO: unexporting *:/volumes/nfs1
exportfs(ExportFS_nfs1)[60393]:	2012/09/10_15:27:23 INFO: unexporting *:/volumes/nfs1
exportfs(ExportFS_nfs1)[60393]:	2012/09/10_15:27:23 INFO: Un-exported file system
exportfs(ExportFS_nfs1)[60393]:	2012/09/10_15:27:23 INFO: Un-exported file system
Sep 10 15:27:23 Cluster-Server-1 lrmd: [48712]: info: Managed ExportFS_nfs1:stop process 60393 exited with return code 0.
Sep 10 15:27:23 Cluster-Server-1 lrmd: [48712]: info: operation stop[46] on ExportFS_nfs1 for client 48715: pid 60393 exited with return code 0
Sep 10 15:27:23 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce ExportFS_nfs1 after complete stop op (interval=0)
Sep 10 15:27:23 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation ExportFS_nfs1_stop_0 (call=46, rc=0, cib-update=89, confirmed=true) ok
Sep 10 15:27:23 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending stop op to history for 'ExportFS_nfs1'
Sep 10 15:27:23 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 160 for pingd=100 passed
Sep 10 15:27:23 Cluster-Server-1 cib: [48709]: info: Managed write_cib_contents process 60392 exited with return code 0.
Sep 10 15:27:23 Cluster-Server-1 crmd: [48715]: debug: cancel_op: Cancelling op 37 for FS_nfs1 (FS_nfs1:37)
Sep 10 15:27:23 Cluster-Server-1 lrmd: [48712]: info: cancel_op: operation monitor[37] on FS_nfs1 for client 48715, its parameters: fstype=[xfs] CRM_meta_timeout=[40000] CRM_meta_name=[monitor] crm_feature_set=[3.0.6] device=[/dev/drive-CSD/nfs1_NFS] force_clones=[false] CRM_meta_interval=[20000] directory=[/volumes/nfs1]  cancelled
Sep 10 15:27:23 Cluster-Server-1 lrmd: [48712]: debug: on_msg_cancel_op: operation 37 cancelled
Sep 10 15:27:23 Cluster-Server-1 crmd: [48715]: debug: cancel_op: Op 37 for FS_nfs1 (FS_nfs1:37): cancelled
Sep 10 15:27:23 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=94:19:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=FS_nfs1_stop_0
Sep 10 15:27:23 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation stop[47] on FS_nfs1 for client 48715, its parameters: crm_feature_set=[3.0.6] CRM_meta_name=[stop] CRM_meta_timeout=[60000]  to the operation list.
Sep 10 15:27:23 Cluster-Server-1 lrmd: [48712]: info: rsc:FS_nfs1 stop[47] (pid 60423)
Sep 10 15:27:23 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation FS_nfs1_monitor_20000 (call=37, status=1, cib-update=0, confirmed=true) Cancelled
Sep 10 15:27:23 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'FS_nfs1'
Filesystem(FS_nfs1)[60423]:	2012/09/10_15:27:23 INFO: Running stop for /dev/drive-CSD/nfs1_NFS on /volumes/nfs1
Filesystem(FS_nfs1)[60423]:	2012/09/10_15:27:23 INFO: Running stop for /dev/drive-CSD/nfs1_NFS on /volumes/nfs1
Filesystem(FS_nfs1)[60423]:	2012/09/10_15:27:23 INFO: Trying to unmount /volumes/nfs1
Filesystem(FS_nfs1)[60423]:	2012/09/10_15:27:23 INFO: Trying to unmount /volumes/nfs1
Filesystem(FS_nfs1)[60423]:	2012/09/10_15:27:23 INFO: unmounted /volumes/nfs1 successfully
Filesystem(FS_nfs1)[60423]:	2012/09/10_15:27:23 INFO: unmounted /volumes/nfs1 successfully
Sep 10 15:27:23 Cluster-Server-1 lrmd: [48712]: info: Managed FS_nfs1:stop process 60423 exited with return code 0.
Sep 10 15:27:23 Cluster-Server-1 lrmd: [48712]: info: operation stop[47] on FS_nfs1 for client 48715: pid 60423 exited with return code 0
Sep 10 15:27:23 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce FS_nfs1 after complete stop op (interval=0)
Sep 10 15:27:23 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation FS_nfs1_stop_0 (call=47, rc=0, cib-update=90, confirmed=true) ok
Sep 10 15:27:23 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending stop op to history for 'FS_nfs1'
Sep 10 15:27:24 Cluster-Server-1 attrd_updater: [60497]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:27:24 Cluster-Server-1 attrd_updater: [60497]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:27:24 Cluster-Server-1 attrd_updater: [60497]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:27:24 Cluster-Server-1 attrd_updater: [60497]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:27:24 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:27:24 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:27:24 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 60314 exited with return code 0
Sep 10 15:27:24 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi3 monitor[31] (pid 60498)
Sep 10 15:27:24 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi3 monitor[33] (pid 60499)
SCSTTarget(Target_iscsi3)[60498]:	2012/09/10_15:27:24 DEBUG: Target_iscsi3 monitor : 0
Sep 10 15:27:24 Cluster-Server-1 lrmd: [48712]: info: operation monitor[31] on Target_iscsi3 for client 48715: pid 60498 exited with return code 0
SCSTLun(Lun_iscsi3)[60499]:	2012/09/10_15:27:24 INFO: Lun_iscsi3 monitor : 0
SCSTLun(Lun_iscsi3)[60499]:	2012/09/10_15:27:24 INFO: Lun_iscsi3 monitor : 0
Sep 10 15:27:24 Cluster-Server-1 lrmd: [48712]: info: operation monitor[33] on Lun_iscsi3 for client 48715: pid 60499 exited with return code 0
Sep 10 15:27:25 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_resource: fail-count-FS_nfs1=<null>
Sep 10 15:27:25 Cluster-Server-1 attrd: [48713]: info: find_hash_entry: Creating hash entry for fail-count-FS_nfs1
Sep 10 15:27:25 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: (null), Current: (null), Stored: (null)
Sep 10 15:27:25 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_resource: fail-count-ExportFS_nfs1=<null>
Sep 10 15:27:25 Cluster-Server-1 attrd: [48713]: info: find_hash_entry: Creating hash entry for fail-count-ExportFS_nfs1
Sep 10 15:27:25 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: (null), Current: (null), Stored: (null)
Sep 10 15:27:25 Cluster-Server-1 cib: [48709]: info: apply_xml_diff: Digest mis-match: expected 6f392ebf34bb36cca0a05e928d37fbd8, calculated 62d2a7ef80be93f5ab8bfad2b36402ee
Sep 10 15:27:25 Cluster-Server-1 cib: [48709]: notice: cib_process_diff: Diff 0.16.8 -> 0.16.9 not applied to 0.16.8: Failed application of an update diff
Sep 10 15:27:25 Cluster-Server-1 cib: [48709]: info: cib_server_process_diff: Requesting re-sync from peer
Sep 10 15:27:25 Cluster-Server-1 crmd: [48715]: info: delete_resource: Removing resource FS_nfs1 for 60562_crm_resource (internal) on Cluster-Server-1
Sep 10 15:27:25 Cluster-Server-1 lrmd: [48712]: debug: lrmd_rsc_destroy: removing resource FS_nfs1
Sep 10 15:27:25 Cluster-Server-1 crmd: [48715]: debug: delete_rsc_entry: sync: Sending delete op for FS_nfs1
Sep 10 15:27:25 Cluster-Server-1 crmd: [48715]: info: notify_deleted: Notifying 60562_crm_resource on Cluster-Server-1 that FS_nfs1 was deleted
Sep 10 15:27:25 Cluster-Server-1 crmd: [48715]: WARN: decode_transition_key: Bad UUID (crm-resource-60562) in sscanf result (3) for 0:0:crm-resource-60562
Sep 10 15:27:25 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: send_direct_ack: Updating resouce FS_nfs1 after complete delete op (interval=60000)
Sep 10 15:27:25 Cluster-Server-1 crmd: [48715]: debug: send_direct_ack: ACK'ing resource op FS_nfs1_delete_60000 from 0:0:crm-resource-60562: lrm_invoke-lrmd-1347283645-37
Sep 10 15:27:25 Cluster-Server-1 crmd: [48715]: debug: notify_deleted: Triggering a refresh after 60562_crm_resource deleted FS_nfs1 from the LRM
Sep 10 15:27:25 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/configuration/crm_config//cluster_property_set//nvpair[@name='last-lrm-refresh'] (/cib/configuration/crm_config/cluster_property_set/nvpair[8])
Sep 10 15:27:25 Cluster-Server-1 crmd: [48715]: debug: find_nvpair_attr_delegate: Match <nvpair id="cib-bootstrap-options-last-lrm-refresh" name="last-lrm-refresh" value="1347283482" />
Sep 10 15:27:25 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for cib_replace op
Sep 10 15:27:25 Cluster-Server-1 cib: [48709]: debug: Forking temp process write_cib_contents
Sep 10 15:27:25 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for cib_apply_diff op
Sep 10 15:27:25 Cluster-Server-1 cib: [48709]: info: apply_xml_diff: Digest mis-match: expected 714169222492ac936c7879de67465f3b, calculated f87c9c03c14cf6d7facc455d6bf62d2d
Sep 10 15:27:25 Cluster-Server-1 cib: [48709]: notice: cib_process_diff: Diff 0.17.1 -> 0.17.2 not applied to 0.17.1: Failed application of an update diff
Sep 10 15:27:25 Cluster-Server-1 cib: [48709]: info: cib_server_process_diff: Requesting re-sync from peer
Sep 10 15:27:25 Cluster-Server-1 crmd: [48715]: info: delete_resource: Removing resource ExportFS_nfs1 for 60562_crm_resource (internal) on Cluster-Server-1
Sep 10 15:27:25 Cluster-Server-1 lrmd: [48712]: debug: lrmd_rsc_destroy: removing resource ExportFS_nfs1
Sep 10 15:27:25 Cluster-Server-1 cib: [48709]: notice: cib_server_process_diff: Not applying diff 0.17.1 -> 0.17.2 (sync in progress)
Sep 10 15:27:25 Cluster-Server-1 crmd: [48715]: debug: delete_rsc_entry: sync: Sending delete op for ExportFS_nfs1
Sep 10 15:27:25 Cluster-Server-1 crmd: [48715]: info: notify_deleted: Notifying 60562_crm_resource on Cluster-Server-1 that ExportFS_nfs1 was deleted
Sep 10 15:27:25 Cluster-Server-1 crmd: [48715]: WARN: decode_transition_key: Bad UUID (crm-resource-60562) in sscanf result (3) for 0:0:crm-resource-60562
Sep 10 15:27:25 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: send_direct_ack: Updating resouce ExportFS_nfs1 after complete delete op (interval=60000)
Sep 10 15:27:25 Cluster-Server-1 crmd: [48715]: debug: send_direct_ack: ACK'ing resource op ExportFS_nfs1_delete_60000 from 0:0:crm-resource-60562: lrm_invoke-lrmd-1347283645-38
Sep 10 15:27:25 Cluster-Server-1 crmd: [48715]: debug: notify_deleted: Triggering a refresh after 60562_crm_resource deleted ExportFS_nfs1 from the LRM
Sep 10 15:27:25 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/configuration/crm_config//cluster_property_set//nvpair[@name='last-lrm-refresh'] (/cib/configuration/crm_config/cluster_property_set/nvpair[8])
Sep 10 15:27:25 Cluster-Server-1 crmd: [48715]: debug: find_nvpair_attr_delegate: Match <nvpair id="cib-bootstrap-options-last-lrm-refresh" name="last-lrm-refresh" value="1347283645" />
Sep 10 15:27:25 Cluster-Server-1 cib: [48709]: info: Managed write_cib_contents process 60563 exited with return code 0.
Sep 10 15:27:25 Cluster-Server-1 cib: [48709]: notice: cib_server_process_diff: Not applying diff 0.17.1 -> 0.17.2 (sync in progress)
Sep 10 15:27:25 Cluster-Server-1 cib: [48709]: debug: Forking temp process write_cib_contents
Sep 10 15:27:25 Cluster-Server-1 cib: [48709]: notice: cib_server_process_diff: Not applying diff 0.17.2 -> 0.17.3 (sync in progress)
Sep 10 15:27:25 Cluster-Server-1 cib: [48709]: notice: cib_server_process_diff: Not applying diff 0.17.3 -> 0.17.4 (sync in progress)
Sep 10 15:27:25 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for cib_replace op
Sep 10 15:27:25 Cluster-Server-1 cib: [48709]: info: cib_replace_notify: Replaced: -1.-1.-1 -> 0.17.3 from Cluster-Server-2
Sep 10 15:27:25 Cluster-Server-1 crmd: [48715]: debug: do_cib_replaced: Updating the CIB after a replace: DC=false
Sep 10 15:27:25 Cluster-Server-1 attrd: [48713]: info: do_cib_replaced: Sending full refresh
Sep 10 15:27:25 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: master-p_Device_drive:0 (10000)
Sep 10 15:27:25 Cluster-Server-1 cib: [48709]: info: Managed write_cib_contents process 60564 exited with return code 0.
Sep 10 15:27:25 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:0'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[3])
Sep 10 15:27:25 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-master-p_Device_drive.0" name="master-p_Device_drive:0" value="10000" />
Sep 10 15:27:25 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: pingd (100)
Sep 10 15:27:25 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
Sep 10 15:27:25 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:27:25 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:27:25 Cluster-Server-1 cib: [48709]: debug: Forking temp process write_cib_contents
Sep 10 15:27:25 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:27:25 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:27:25 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 162 for master-p_Device_drive:0=10000 passed
Sep 10 15:27:25 Cluster-Server-1 lrmd: [48712]: debug: on_msg_add_rsc:client [48715] adds resource FS_nfs1
Sep 10 15:27:25 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=18:21:7:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=FS_nfs1_monitor_0
Sep 10 15:27:25 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc FS_nfs1
Sep 10 15:27:25 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[48] on FS_nfs1 for client 48715, its parameters: crm_feature_set=[3.0.6] device=[/dev/drive-CSD/nfs1_NFS] directory=[/volumes/nfs1] force_clones=[false] fstype=[xfs] CRM_meta_timeout=[20000]  to the operation list.
Sep 10 15:27:25 Cluster-Server-1 lrmd: [48712]: info: rsc:FS_nfs1 probe[48] (pid 60583)
Sep 10 15:27:25 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 164 for probe_complete=true passed
Sep 10 15:27:25 Cluster-Server-1 lrmd: [48712]: debug: on_msg_add_rsc:client [48715] adds resource ExportFS_nfs1
Sep 10 15:27:25 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=19:21:7:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=ExportFS_nfs1_monitor_0
Sep 10 15:27:25 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc ExportFS_nfs1
Sep 10 15:27:25 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[49] on ExportFS_nfs1 for client 48715, its parameters: crm_feature_set=[3.0.6] options=[rw,insecure,async,no_subtree_check,root_squash,no_all_squash,anonuid=1directory=[/volumes/nfs1] fsid=[1955f364-fb4b-11e1-b02e-000c290247c7] clientspec=[*] CRM_meta_timeout=[20000]  to the operation list.
Sep 10 15:27:25 Cluster-Server-1 lrmd: [48712]: info: rsc:ExportFS_nfs1 probe[49] (pid 60586)
Sep 10 15:27:25 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 166 for pingd=100 passed
exportfs(ExportFS_nfs1)[60586]:	2012/09/10_15:27:25 INFO: Directory /volumes/nfs1 is not exported to * (stopped).
exportfs(ExportFS_nfs1)[60586]:	2012/09/10_15:27:25 INFO: Directory /volumes/nfs1 is not exported to * (stopped).
Sep 10 15:27:25 Cluster-Server-1 lrmd: [48712]: WARN: Managed ExportFS_nfs1:monitor process 60586 exited with return code 7.
Sep 10 15:27:25 Cluster-Server-1 lrmd: [48712]: info: operation monitor[49] on ExportFS_nfs1 for client 48715: pid 60586 exited with return code 7
Sep 10 15:27:25 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce ExportFS_nfs1 after complete monitor op (interval=0)
Sep 10 15:27:25 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation ExportFS_nfs1_monitor_0 (call=49, rc=7, cib-update=99, confirmed=true) not running
Sep 10 15:27:25 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'ExportFS_nfs1'
Sep 10 15:27:25 Cluster-Server-1 lrmd: [48712]: WARN: Managed FS_nfs1:monitor process 60583 exited with return code 7.
Sep 10 15:27:25 Cluster-Server-1 lrmd: [48712]: info: operation monitor[48] on FS_nfs1 for client 48715: pid 60583 exited with return code 7
Sep 10 15:27:25 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce FS_nfs1 after complete monitor op (interval=0)
Sep 10 15:27:25 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation FS_nfs1_monitor_0 (call=48, rc=7, cib-update=100, confirmed=true) not running
Sep 10 15:27:25 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'FS_nfs1'
Sep 10 15:27:25 Cluster-Server-1 cib: [48709]: info: Managed write_cib_contents process 60572 exited with return code 0.
Sep 10 15:27:25 Cluster-Server-1 crmd: [48715]: debug: attrd_update_delegate: Sent update: probe_complete=true for localhost
Sep 10 15:27:25 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crmd: probe_complete=true
Sep 10 15:27:25 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: true, Current: true, Stored: true
Sep 10 15:27:25 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi2 monitor[21] (pid 60647)
Sep 10 15:27:25 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi2 monitor[23] (pid 60652)
SCSTTarget(Target_iscsi2)[60647]:	2012/09/10_15:27:25 DEBUG: Target_iscsi2 monitor : 0
Sep 10 15:27:25 Cluster-Server-1 lrmd: [48712]: info: operation monitor[21] on Target_iscsi2 for client 48715: pid 60647 exited with return code 0
SCSTLun(Lun_iscsi2)[60652]:	2012/09/10_15:27:25 INFO: Lun_iscsi2 monitor : 0
SCSTLun(Lun_iscsi2)[60652]:	2012/09/10_15:27:25 INFO: Lun_iscsi2 monitor : 0
Sep 10 15:27:25 Cluster-Server-1 lrmd: [48712]: info: operation monitor[23] on Lun_iscsi2 for client 48715: pid 60652 exited with return code 0
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [60677] registered
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:60677] disconnected.
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:60677] is unregistered
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [60679] registered
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:60679] disconnected.
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:60679] is unregistered
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [60681] registered
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:60681] disconnected.
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:60681] is unregistered
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [60683] registered
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:60683] disconnected.
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:60683] is unregistered
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [60692] registered
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:60692] disconnected.
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:60692] is unregistered
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [60701] registered
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:60701] disconnected.
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:60701] is unregistered
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [60708] registered
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:60708] disconnected.
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:60708] is unregistered
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [60715] registered
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:60715] disconnected.
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:60715] is unregistered
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [60722] registered
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:60722] disconnected.
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:60722] is unregistered
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [60730] registered
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:60730] disconnected.
Sep 10 15:27:26 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:60730] is unregistered
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_COUNT
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: crm_uptime: Current CPU usage is: 0s, 120000us
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: crm_compare_age: Loose: 120000 vs 520000 (usec)
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: info: do_election_count_vote: Election 13 (owner: Cluster-Server-2) lost: vote from Cluster-Server-2 (Uptime)
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: update_dc: Unset DC. Was Cluster-Server-2
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_CHECK
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: do_election_check: Ignore election check: we not in an election
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_PENDING: [ state=S_NOT_DC cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_LOG   
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: do_log: FSA: Input I_PENDING from do_election_count_vote() received in state S_NOT_DC
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: notice: do_state_transition: State transition S_NOT_DC -> S_PENDING [ input=I_PENDING cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_INTEGRATE_TIMER_STOP
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_FINALIZE_TIMER_STOP
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_START
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: crm_timer_start: Started Election Trigger (I_DC_TIMEOUT:20000ms), src=89
Sep 10 15:27:26 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for cib_apply_diff op
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: handle_request: Raising I_JOIN_OFFER: join-11
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_JOIN_OFFER: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ]
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_CL_JOIN_REQUEST
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: info: update_dc: Set DC to Cluster-Server-2 (3.0.6)
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_offer_respond: do_cl_join_offer_respond added action A_DC_TIMER_STOP to the FSA
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:27:26 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:1'] does not exist
Sep 10 15:27:26 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update -22 for master-p_Device_drive:1=(null) passed
Sep 10 15:27:26 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:27:26 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:27:26 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:27:26 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:27:26 Cluster-Server-1 cib: [48709]: debug: Forking temp process write_cib_contents
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: join_query_callback: Respond to join offer join-11
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: join_query_callback: Acknowledging Cluster-Server-2 as our DC
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: handle_request: Raising I_JOIN_RESULT: join-11
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_JOIN_RESULT: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ]
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_CL_JOIN_RESULT
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_finalize_respond: Confirming join join-11: join_ack_nack
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce FS_nfs2 after complete start op (interval=0)
Sep 10 15:27:26 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 169 for pingd=100 passed
Sep 10 15:27:26 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 171 for probe_complete=true passed
Sep 10 15:27:26 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for cib_replace op
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce FS_nfs2 after complete monitor op (interval=20000)
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce ExportFS_nfs1 after complete monitor op (interval=0)
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce ExportFS_nfs2 after complete start op (interval=0)
Sep 10 15:27:26 Cluster-Server-1 cib: [48709]: info: Managed write_cib_contents process 60742 exited with return code 0.
Sep 10 15:27:26 Cluster-Server-1 cib: [48709]: debug: Forking temp process write_cib_contents
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce ExportFS_nfs2 after complete monitor op (interval=10000)
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_NFS_Server:0 after complete start op (interval=0)
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_NFS_Server:0 after complete monitor op (interval=30000)
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce FS_nfs1 after complete monitor op (interval=0)
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce LVM_drive after complete monitor op (interval=0)
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Lun_iscsi1 after complete monitor op (interval=0)
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_PingD:0 after complete monitor op (interval=0)
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_PingD:0 after complete monitor op (interval=10000)
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Lun_iscsi2 after complete start op (interval=0)
Sep 10 15:27:26 Cluster-Server-1 cib: [48709]: info: Managed write_cib_contents process 60755 exited with return code 0.
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Lun_iscsi2 after complete monitor op (interval=10000)
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_Device_drive:0 after complete monitor op (interval=0)
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_Device_drive:0 after complete monitor op (interval=10000)
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_iSCSI_Daemon:0 after complete monitor op (interval=0)
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_iSCSI_Daemon:0 after complete monitor op (interval=30000)
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Target_iscsi2 after complete start op (interval=0)
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Target_iscsi2 after complete monitor op (interval=10000)
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Target_iscsi3 after complete start op (interval=0)
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Target_iscsi3 after complete monitor op (interval=10000)
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Target_iscsi1 after complete monitor op (interval=0)
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Lun_iscsi3 after complete start op (interval=0)
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Lun_iscsi3 after complete monitor op (interval=10000)
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_finalize_respond: join-11: Join complete.  Sending local LRM status to Cluster-Server-2
Sep 10 15:27:26 Cluster-Server-1 attrd: [48713]: notice: attrd_local_callback: Sending full refresh (origin=crmd)
Sep 10 15:27:26 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: master-p_Device_drive:0 (10000)
Sep 10 15:27:26 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:0'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[3])
Sep 10 15:27:26 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-master-p_Device_drive.0" name="master-p_Device_drive:0" value="10000" />
Sep 10 15:27:26 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: pingd (100)
Sep 10 15:27:26 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
Sep 10 15:27:26 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: attrd_update_delegate: Sent update: (null)=(null) for localhost
Sep 10 15:27:26 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_NOT_DC: [ state=S_PENDING cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_LOG   
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: do_log: FSA: Input I_NOT_DC from do_cl_join_finalize_respond() received in state S_PENDING
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: notice: do_state_transition: State transition S_PENDING -> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_INTEGRATE_TIMER_STOP
Sep 10 15:27:26 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_FINALIZE_TIMER_STOP
Sep 10 15:27:26 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:27:26 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:27:26 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 173 for master-p_Device_drive:0=10000 passed
Sep 10 15:27:26 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 175 for probe_complete=true passed
Sep 10 15:27:26 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 177 for pingd=100 passed
Sep 10 15:27:26 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:1'] does not exist
Sep 10 15:27:26 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update -22 for master-p_Device_drive:1=(null) passed
Sep 10 15:27:26 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:27:26 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:27:26 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:27:26 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:27:26 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 180 for pingd=100 passed
Sep 10 15:27:26 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 182 for probe_complete=true passed
Sep 10 15:27:29 Cluster-Server-1 lrmd: [48712]: debug: rsc:FS_nfs2 monitor[43] (pid 60787)
Sep 10 15:27:29 Cluster-Server-1 lrmd: [48712]: info: operation monitor[43] on FS_nfs2 for client 48715: pid 60787 exited with return code 0
Sep 10 15:27:29 Cluster-Server-1 lrmd: [48712]: debug: rsc:ExportFS_nfs2 monitor[45] (pid 60819)
exportfs(ExportFS_nfs2)[60819]:	2012/09/10_15:27:29 INFO: Directory /volumes/nfs2 is exported to * (started).
exportfs(ExportFS_nfs2)[60819]:	2012/09/10_15:27:29 INFO: Directory /volumes/nfs2 is exported to * (started).
Sep 10 15:27:29 Cluster-Server-1 lrmd: [48712]: info: operation monitor[45] on ExportFS_nfs2 for client 48715: pid 60819 exited with return code 0
Sep 10 15:27:29 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 60832)
drbd(p_Device_drive:0)[60832]:	2012/09/10_15:27:29 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:27:29 Cluster-Server-1 crm_attribute: [60862]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:27:29 Cluster-Server-1 crm_attribute: [60862]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:27:29 Cluster-Server-1 crm_attribute: [60862]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:27:29 Cluster-Server-1 crm_attribute: [60862]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:27:29 Cluster-Server-1 crm_attribute: [60862]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:27:29 Cluster-Server-1 crm_attribute: [60862]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:27:29 Cluster-Server-1 crm_attribute: [60862]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:27:29 Cluster-Server-1 crm_attribute: [60862]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:27:29 Cluster-Server-1 crm_attribute: [60862]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:27:29 Cluster-Server-1 crm_attribute: [60862]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:27:29 Cluster-Server-1 crm_attribute: [60862]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:27:29 Cluster-Server-1 crm_attribute: [60862]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:27:29 Cluster-Server-1 crm_attribute: [60862]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:27:29 Cluster-Server-1 crm_attribute: [60862]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:27:29 Cluster-Server-1 crm_attribute: [60862]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:27:29 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:27:29 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[60832]:	2012/09/10_15:27:29 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[60832]:	2012/09/10_15:27:29 DEBUG: drive: Command output: 
Sep 10 15:27:29 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:27:29 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 60832 exited with return code 8
Sep 10 15:27:34 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 60878)
Sep 10 15:27:34 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi3 monitor[31] (pid 61028)
Sep 10 15:27:34 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi3 monitor[33] (pid 61029)
SCSTTarget(Target_iscsi3)[61028]:	2012/09/10_15:27:34 DEBUG: Target_iscsi3 monitor : 0
Sep 10 15:27:34 Cluster-Server-1 lrmd: [48712]: info: operation monitor[31] on Target_iscsi3 for client 48715: pid 61028 exited with return code 0
SCSTLun(Lun_iscsi3)[61029]:	2012/09/10_15:27:34 INFO: Lun_iscsi3 monitor : 0
SCSTLun(Lun_iscsi3)[61029]:	2012/09/10_15:27:34 INFO: Lun_iscsi3 monitor : 0
Sep 10 15:27:34 Cluster-Server-1 lrmd: [48712]: info: operation monitor[33] on Lun_iscsi3 for client 48715: pid 61029 exited with return code 0
Sep 10 15:27:35 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi2 monitor[21] (pid 61104)
Sep 10 15:27:35 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi2 monitor[23] (pid 61105)
SCSTTarget(Target_iscsi2)[61104]:	2012/09/10_15:27:35 DEBUG: Target_iscsi2 monitor : 0
Sep 10 15:27:35 Cluster-Server-1 lrmd: [48712]: info: operation monitor[21] on Target_iscsi2 for client 48715: pid 61104 exited with return code 0
SCSTLun(Lun_iscsi2)[61105]:	2012/09/10_15:27:35 INFO: Lun_iscsi2 monitor : 0
SCSTLun(Lun_iscsi2)[61105]:	2012/09/10_15:27:35 INFO: Lun_iscsi2 monitor : 0
Sep 10 15:27:35 Cluster-Server-1 lrmd: [48712]: info: operation monitor[23] on Lun_iscsi2 for client 48715: pid 61105 exited with return code 0
Sep 10 15:27:36 Cluster-Server-1 attrd_updater: [61120]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:27:36 Cluster-Server-1 attrd_updater: [61120]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:27:36 Cluster-Server-1 attrd_updater: [61120]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:27:36 Cluster-Server-1 attrd_updater: [61120]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:27:36 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:27:36 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:27:36 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 60878 exited with return code 0
Sep 10 15:27:39 Cluster-Server-1 lrmd: [48712]: debug: rsc:ExportFS_nfs2 monitor[45] (pid 61121)
exportfs(ExportFS_nfs2)[61121]:	2012/09/10_15:27:39 INFO: Directory /volumes/nfs2 is exported to * (started).
exportfs(ExportFS_nfs2)[61121]:	2012/09/10_15:27:39 INFO: Directory /volumes/nfs2 is exported to * (started).
Sep 10 15:27:39 Cluster-Server-1 lrmd: [48712]: info: operation monitor[45] on ExportFS_nfs2 for client 48715: pid 61121 exited with return code 0
Sep 10 15:27:39 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 61132)
drbd(p_Device_drive:0)[61132]:	2012/09/10_15:27:39 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:27:39 Cluster-Server-1 crm_attribute: [61162]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:27:39 Cluster-Server-1 crm_attribute: [61162]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:27:39 Cluster-Server-1 crm_attribute: [61162]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:27:39 Cluster-Server-1 crm_attribute: [61162]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:27:39 Cluster-Server-1 crm_attribute: [61162]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:27:39 Cluster-Server-1 crm_attribute: [61162]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:27:39 Cluster-Server-1 crm_attribute: [61162]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:27:39 Cluster-Server-1 crm_attribute: [61162]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:27:39 Cluster-Server-1 crm_attribute: [61162]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:27:39 Cluster-Server-1 crm_attribute: [61162]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:27:39 Cluster-Server-1 crm_attribute: [61162]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:27:39 Cluster-Server-1 crm_attribute: [61162]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:27:39 Cluster-Server-1 crm_attribute: [61162]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:27:39 Cluster-Server-1 crm_attribute: [61162]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:27:39 Cluster-Server-1 crm_attribute: [61162]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:27:39 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:27:39 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[61132]:	2012/09/10_15:27:39 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[61132]:	2012/09/10_15:27:39 DEBUG: drive: Command output: 
Sep 10 15:27:39 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:27:39 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 61132 exited with return code 8
Sep 10 15:27:44 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi3 monitor[31] (pid 61471)
Sep 10 15:27:44 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi3 monitor[33] (pid 61472)
SCSTTarget(Target_iscsi3)[61471]:	2012/09/10_15:27:44 DEBUG: Target_iscsi3 monitor : 0
Sep 10 15:27:44 Cluster-Server-1 lrmd: [48712]: info: operation monitor[31] on Target_iscsi3 for client 48715: pid 61471 exited with return code 0
SCSTLun(Lun_iscsi3)[61472]:	2012/09/10_15:27:44 INFO: Lun_iscsi3 monitor : 0
SCSTLun(Lun_iscsi3)[61472]:	2012/09/10_15:27:44 INFO: Lun_iscsi3 monitor : 0
Sep 10 15:27:44 Cluster-Server-1 lrmd: [48712]: info: operation monitor[33] on Lun_iscsi3 for client 48715: pid 61472 exited with return code 0
Sep 10 15:27:45 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_NFS_Server:0 monitor[8] (pid 61490)
Sep 10 15:27:45 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_iSCSI_Daemon:0 monitor[5] (pid 61491)
Sep 10 15:27:45 Cluster-Server-1 lrmd: [48712]: info: operation monitor[8] on p_NFS_Server:0 for client 48715: pid 61490 exited with return code 0
Sep 10 15:27:45 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_NFS_Server:0:monitor:stdout) nfsd running

Sep 10 15:27:45 Cluster-Server-1 lrmd: [48712]: info: operation monitor[5] on p_iSCSI_Daemon:0 for client 48715: pid 61491 exited with return code 0
Sep 10 15:27:45 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi2 monitor[21] (pid 61496)
Sep 10 15:27:45 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi2 monitor[23] (pid 61497)
SCSTTarget(Target_iscsi2)[61496]:	2012/09/10_15:27:45 DEBUG: Target_iscsi2 monitor : 0
Sep 10 15:27:45 Cluster-Server-1 lrmd: [48712]: info: operation monitor[21] on Target_iscsi2 for client 48715: pid 61496 exited with return code 0
SCSTLun(Lun_iscsi2)[61497]:	2012/09/10_15:27:45 INFO: Lun_iscsi2 monitor : 0
SCSTLun(Lun_iscsi2)[61497]:	2012/09/10_15:27:45 INFO: Lun_iscsi2 monitor : 0
Sep 10 15:27:45 Cluster-Server-1 lrmd: [48712]: info: operation monitor[23] on Lun_iscsi2 for client 48715: pid 61497 exited with return code 0
Sep 10 15:27:46 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 61560)
Sep 10 15:27:46 Cluster-Server-1 cib: [48709]: info: apply_xml_diff: Digest mis-match: expected 91853bf8a4acb53804b862712b09eb6a, calculated b2384384dcdaa2517c6d1fc95672d25a
Sep 10 15:27:46 Cluster-Server-1 cib: [48709]: notice: cib_process_diff: Diff 0.18.31 -> 0.19.1 not applied to 0.18.31: Failed application of an update diff
Sep 10 15:27:46 Cluster-Server-1 cib: [48709]: info: cib_server_process_diff: Requesting re-sync from peer
Sep 10 15:27:46 Cluster-Server-1 cib: [48709]: debug: activateCibXml: Triggering CIB write for cib_replace op
Sep 10 15:27:46 Cluster-Server-1 cib: [48709]: info: cib_replace_notify: Replaced: -1.-1.-1 -> 0.19.1 from Cluster-Server-2
Sep 10 15:27:46 Cluster-Server-1 crmd: [48715]: debug: do_cib_replaced: Updating the CIB after a replace: DC=false
Sep 10 15:27:46 Cluster-Server-1 attrd: [48713]: info: do_cib_replaced: Sending full refresh
Sep 10 15:27:46 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: master-p_Device_drive:0 (10000)
Sep 10 15:27:46 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:0'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[3])
Sep 10 15:27:46 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-master-p_Device_drive.0" name="master-p_Device_drive:0" value="10000" />
Sep 10 15:27:46 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: pingd (100)
Sep 10 15:27:46 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
Sep 10 15:27:46 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:27:46 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:27:46 Cluster-Server-1 cib: [48709]: debug: Forking temp process write_cib_contents
Sep 10 15:27:46 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:27:46 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:27:46 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 184 for master-p_Device_drive:0=10000 passed
Sep 10 15:27:46 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 186 for probe_complete=true passed
Sep 10 15:27:46 Cluster-Server-1 cib: [61587]: ERROR: validate_cib_digest: Digest comparision failed: expected 915815b4e68298e50146ce8cc380b8d9 (/var/lib/heartbeat/crm/cib.4Fs3XB), calculated 0121bf5cffe4dab4f888959f1baf7322
Sep 10 15:27:46 Cluster-Server-1 cib: [61587]: ERROR: retrieveCib: Checksum of /var/lib/heartbeat/crm/cib.CcXphf failed!  Configuration contents ignored!
Sep 10 15:27:46 Cluster-Server-1 cib: [61587]: ERROR: retrieveCib: Usually this is caused by manual changes, please refer to http://clusterlabs.org/wiki/FAQ#cib_changes_detected
Sep 10 15:27:46 Cluster-Server-1 cib: [61587]: ERROR: crm_abort: write_cib_contents: Triggered fatal assert at io.c:662 : retrieveCib(tmp1, tmp2, FALSE) != NULL
Sep 10 15:27:46 Cluster-Server-1 crmd: [48715]: debug: cancel_op: Cancelling op 33 for Lun_iscsi3 (Lun_iscsi3:33)
Sep 10 15:27:46 Cluster-Server-1 lrmd: [48712]: info: cancel_op: operation monitor[33] on Lun_iscsi3 for client 48715, its parameters: handler=[vdisk_blockio] CRM_meta_timeout=[20000] CRM_meta_name=[monitor] target_iqn=[iqn.2005-07.com.example:vdisk.iscsi3] path=[/dev/drive-CSD/iscsi3_iSCSI] crm_feature_set=[3.0.6] CRM_meta_interval=[10000] lun=[0] device_name=[iscsi3]  cancelled
Sep 10 15:27:46 Cluster-Server-1 lrmd: [48712]: debug: on_msg_cancel_op: operation 33 cancelled
Sep 10 15:27:46 Cluster-Server-1 crmd: [48715]: debug: cancel_op: Op 33 for Lun_iscsi3 (Lun_iscsi3:33): cancelled
Sep 10 15:27:46 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=85:24:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=Lun_iscsi3_stop_0
Sep 10 15:27:46 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation stop[50] on Lun_iscsi3 for client 48715, its parameters: crm_feature_set=[3.0.6] CRM_meta_name=[stop] CRM_meta_timeout=[240000]  to the operation list.
Sep 10 15:27:46 Cluster-Server-1 lrmd: [48712]: info: rsc:Lun_iscsi3 stop[50] (pid 61589)
Sep 10 15:27:46 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Lun_iscsi3_monitor_10000 (call=33, status=1, cib-update=0, confirmed=true) Cancelled
Sep 10 15:27:46 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'Lun_iscsi3'
Sep 10 15:27:46 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 188 for pingd=100 passed
Sep 10 15:27:46 Cluster-Server-1 cib: [48709]: WARN: Managed write_cib_contents process 61587 killed by signal 6 [SIGABRT - Abort].
Sep 10 15:27:46 Cluster-Server-1 cib: [48709]: ERROR: Managed write_cib_contents process 61587 dumped core
Sep 10 15:27:46 Cluster-Server-1 cib: [48709]: ERROR: cib_diskwrite_complete: Disk write failed: status=134, signo=6, exitcode=0
Sep 10 15:27:46 Cluster-Server-1 cib: [48709]: ERROR: cib_diskwrite_complete: Disabling disk writes after write failure
SCSTLun(Lun_iscsi3)[61589]:	2012/09/10_15:27:46 INFO: Stopping lun 0 on target iqn.2005-07.com.example:vdisk.iscsi3
SCSTLun(Lun_iscsi3)[61589]:	2012/09/10_15:27:46 INFO: Stopping lun 0 on target iqn.2005-07.com.example:vdisk.iscsi3
SCSTLun(Lun_iscsi3)[61589]:	2012/09/10_15:27:46 INFO: Disabling target iqn.2005-07.com.example:vdisk.iscsi3
SCSTLun(Lun_iscsi3)[61589]:	2012/09/10_15:27:46 INFO: Disabling target iqn.2005-07.com.example:vdisk.iscsi3
SCSTLun(Lun_iscsi3)[61589]:	2012/09/10_15:27:46 INFO: Removing LUN 0, device iscsi3, target iqn.2005-07.com.example:vdisk.iscsi3
SCSTLun(Lun_iscsi3)[61589]:	2012/09/10_15:27:46 INFO: Removing LUN 0, device iscsi3, target iqn.2005-07.com.example:vdisk.iscsi3
SCSTLun(Lun_iscsi3)[61589]:	2012/09/10_15:27:46 INFO: Closing device iscsi3
SCSTLun(Lun_iscsi3)[61589]:	2012/09/10_15:27:46 INFO: Closing device iscsi3
SCSTLun(Lun_iscsi3)[61589]:	2012/09/10_15:27:46 INFO: Enabling target iqn.2005-07.com.example:vdisk.iscsi3
SCSTLun(Lun_iscsi3)[61589]:	2012/09/10_15:27:46 INFO: Enabling target iqn.2005-07.com.example:vdisk.iscsi3
SCSTLun(Lun_iscsi3)[61589]:	2012/09/10_15:27:46 INFO: Lun_iscsi3 stop : 0
SCSTLun(Lun_iscsi3)[61589]:	2012/09/10_15:27:46 INFO: Lun_iscsi3 stop : 0
Sep 10 15:27:46 Cluster-Server-1 lrmd: [48712]: info: Managed Lun_iscsi3:stop process 61589 exited with return code 0.
Sep 10 15:27:46 Cluster-Server-1 lrmd: [48712]: info: operation stop[50] on Lun_iscsi3 for client 48715: pid 61589 exited with return code 0
Sep 10 15:27:46 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce Lun_iscsi3 after complete stop op (interval=0)
Sep 10 15:27:46 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Lun_iscsi3_stop_0 (call=50, rc=0, cib-update=104, confirmed=true) ok
Sep 10 15:27:46 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending stop op to history for 'Lun_iscsi3'
Sep 10 15:27:46 Cluster-Server-1 crmd: [48715]: debug: cancel_op: Cancelling op 31 for Target_iscsi3 (Target_iscsi3:31)
Sep 10 15:27:46 Cluster-Server-1 lrmd: [48712]: info: cancel_op: operation monitor[31] on Target_iscsi3 for client 48715, its parameters: crm_feature_set=[3.0.6] CRM_meta_name=[monitor] iqn=[iqn.2005-07.com.example:vdisk.iscsi3] CRM_meta_timeout=[60000] CRM_meta_interval=[10000]  cancelled
Sep 10 15:27:46 Cluster-Server-1 lrmd: [48712]: debug: on_msg_cancel_op: operation 31 cancelled
Sep 10 15:27:46 Cluster-Server-1 crmd: [48715]: debug: cancel_op: Op 31 for Target_iscsi3 (Target_iscsi3:31): cancelled
Sep 10 15:27:46 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=84:24:0:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=Target_iscsi3_stop_0
Sep 10 15:27:46 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation stop[51] on Target_iscsi3 for client 48715, its parameters: crm_feature_set=[3.0.6] CRM_meta_name=[stop] CRM_meta_timeout=[240000]  to the operation list.
Sep 10 15:27:46 Cluster-Server-1 lrmd: [48712]: info: rsc:Target_iscsi3 stop[51] (pid 61627)
Sep 10 15:27:46 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Target_iscsi3_monitor_10000 (call=31, status=1, cib-update=0, confirmed=true) Cancelled
Sep 10 15:27:46 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'Target_iscsi3'
SCSTTarget(Target_iscsi3)[61627]:	2012/09/10_15:27:46 INFO: target iqn.2005-07.com.example:vdisk.iscsi3: Stopping...
SCSTTarget(Target_iscsi3)[61627]:	2012/09/10_15:27:46 INFO: target iqn.2005-07.com.example:vdisk.iscsi3: Stopping...
SCSTTarget(Target_iscsi3)[61627]:	2012/09/10_15:27:46 INFO: disabling target iqn.2005-07.com.example:vdisk.iscsi3
SCSTTarget(Target_iscsi3)[61627]:	2012/09/10_15:27:46 INFO: disabling target iqn.2005-07.com.example:vdisk.iscsi3
SCSTTarget(Target_iscsi3)[61627]:	2012/09/10_15:27:46 INFO: deleting target iqn.2005-07.com.example:vdisk.iscsi3
SCSTTarget(Target_iscsi3)[61627]:	2012/09/10_15:27:46 INFO: deleting target iqn.2005-07.com.example:vdisk.iscsi3
SCSTTarget(Target_iscsi3)[61627]:	2012/09/10_15:27:46 INFO: target iqn.2005-07.com.example:vdisk.iscsi3: Stopped.
SCSTTarget(Target_iscsi3)[61627]:	2012/09/10_15:27:46 INFO: target iqn.2005-07.com.example:vdisk.iscsi3: Stopped.
SCSTTarget(Target_iscsi3)[61627]:	2012/09/10_15:27:46 DEBUG: Target_iscsi3 stop : 0
Sep 10 15:27:46 Cluster-Server-1 lrmd: [48712]: info: Managed Target_iscsi3:stop process 61627 exited with return code 0.
Sep 10 15:27:46 Cluster-Server-1 lrmd: [48712]: info: operation stop[51] on Target_iscsi3 for client 48715: pid 61627 exited with return code 0
Sep 10 15:27:46 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce Target_iscsi3 after complete stop op (interval=0)
Sep 10 15:27:46 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Target_iscsi3_stop_0 (call=51, rc=0, cib-update=105, confirmed=true) ok
Sep 10 15:27:46 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending stop op to history for 'Target_iscsi3'
Sep 10 15:27:48 Cluster-Server-1 attrd_updater: [61666]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:27:48 Cluster-Server-1 attrd_updater: [61666]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:27:48 Cluster-Server-1 attrd_updater: [61666]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:27:48 Cluster-Server-1 attrd_updater: [61666]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:27:48 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:27:48 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:27:48 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 61560 exited with return code 0
Sep 10 15:27:48 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_resource: fail-count-Target_iscsi3=<null>
Sep 10 15:27:48 Cluster-Server-1 attrd: [48713]: info: find_hash_entry: Creating hash entry for fail-count-Target_iscsi3
Sep 10 15:27:48 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: (null), Current: (null), Stored: (null)
Sep 10 15:27:48 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_resource: fail-count-Lun_iscsi3=<null>
Sep 10 15:27:48 Cluster-Server-1 attrd: [48713]: info: find_hash_entry: Creating hash entry for fail-count-Lun_iscsi3
Sep 10 15:27:48 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: (null), Current: (null), Stored: (null)
Sep 10 15:27:48 Cluster-Server-1 cib: [48709]: info: apply_xml_diff: Digest mis-match: expected 9b4ba835a224eb572f4b072ce66576c0, calculated 149597b6959d5b40e81a6b0cfd00bd4c
Sep 10 15:27:48 Cluster-Server-1 cib: [48709]: notice: cib_process_diff: Diff 0.19.8 -> 0.19.9 not applied to 0.19.8: Failed application of an update diff
Sep 10 15:27:48 Cluster-Server-1 cib: [48709]: info: cib_server_process_diff: Requesting re-sync from peer
Sep 10 15:27:48 Cluster-Server-1 crmd: [48715]: info: delete_resource: Removing resource Target_iscsi3 for 61717_crm_resource (internal) on Cluster-Server-1
Sep 10 15:27:48 Cluster-Server-1 lrmd: [48712]: debug: lrmd_rsc_destroy: removing resource Target_iscsi3
Sep 10 15:27:48 Cluster-Server-1 crmd: [48715]: debug: delete_rsc_entry: sync: Sending delete op for Target_iscsi3
Sep 10 15:27:48 Cluster-Server-1 crmd: [48715]: info: notify_deleted: Notifying 61717_crm_resource on Cluster-Server-1 that Target_iscsi3 was deleted
Sep 10 15:27:48 Cluster-Server-1 crmd: [48715]: WARN: decode_transition_key: Bad UUID (crm-resource-61717) in sscanf result (3) for 0:0:crm-resource-61717
Sep 10 15:27:48 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: send_direct_ack: Updating resouce Target_iscsi3 after complete delete op (interval=60000)
Sep 10 15:27:48 Cluster-Server-1 crmd: [48715]: debug: send_direct_ack: ACK'ing resource op Target_iscsi3_delete_60000 from 0:0:crm-resource-61717: lrm_invoke-lrmd-1347283668-43
Sep 10 15:27:48 Cluster-Server-1 crmd: [48715]: debug: notify_deleted: Triggering a refresh after 61717_crm_resource deleted Target_iscsi3 from the LRM
Sep 10 15:27:48 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/configuration/crm_config//cluster_property_set//nvpair[@name='last-lrm-refresh'] (/cib/configuration/crm_config/cluster_property_set/nvpair[8])
Sep 10 15:27:48 Cluster-Server-1 crmd: [48715]: debug: find_nvpair_attr_delegate: Match <nvpair id="cib-bootstrap-options-last-lrm-refresh" name="last-lrm-refresh" value="1347283645" />
Sep 10 15:27:48 Cluster-Server-1 cib: [48709]: info: apply_xml_diff: Digest mis-match: expected e3dfc19c398b1993cd25df90233a259e, calculated 1bc6a11a9f3debfd056f550f494795ad
Sep 10 15:27:48 Cluster-Server-1 cib: [48709]: notice: cib_process_diff: Diff 0.20.1 -> 0.20.2 not applied to 0.20.1: Failed application of an update diff
Sep 10 15:27:48 Cluster-Server-1 cib: [48709]: info: cib_server_process_diff: Requesting re-sync from peer
Sep 10 15:27:48 Cluster-Server-1 crmd: [48715]: info: delete_resource: Removing resource Lun_iscsi3 for 61717_crm_resource (internal) on Cluster-Server-1
Sep 10 15:27:48 Cluster-Server-1 cib: [48709]: notice: cib_server_process_diff: Not applying diff 0.20.1 -> 0.20.2 (sync in progress)
Sep 10 15:27:48 Cluster-Server-1 lrmd: [48712]: debug: lrmd_rsc_destroy: removing resource Lun_iscsi3
Sep 10 15:27:48 Cluster-Server-1 crmd: [48715]: debug: delete_rsc_entry: sync: Sending delete op for Lun_iscsi3
Sep 10 15:27:48 Cluster-Server-1 crmd: [48715]: info: notify_deleted: Notifying 61717_crm_resource on Cluster-Server-1 that Lun_iscsi3 was deleted
Sep 10 15:27:48 Cluster-Server-1 crmd: [48715]: WARN: decode_transition_key: Bad UUID (crm-resource-61717) in sscanf result (3) for 0:0:crm-resource-61717
Sep 10 15:27:48 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: send_direct_ack: Updating resouce Lun_iscsi3 after complete delete op (interval=60000)
Sep 10 15:27:48 Cluster-Server-1 crmd: [48715]: debug: send_direct_ack: ACK'ing resource op Lun_iscsi3_delete_60000 from 0:0:crm-resource-61717: lrm_invoke-lrmd-1347283668-44
Sep 10 15:27:48 Cluster-Server-1 crmd: [48715]: debug: notify_deleted: Triggering a refresh after 61717_crm_resource deleted Lun_iscsi3 from the LRM
Sep 10 15:27:48 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/configuration/crm_config//cluster_property_set//nvpair[@name='last-lrm-refresh'] (/cib/configuration/crm_config/cluster_property_set/nvpair[8])
Sep 10 15:27:48 Cluster-Server-1 crmd: [48715]: debug: find_nvpair_attr_delegate: Match <nvpair id="cib-bootstrap-options-last-lrm-refresh" name="last-lrm-refresh" value="1347283668" />
Sep 10 15:27:48 Cluster-Server-1 cib: [48709]: notice: cib_server_process_diff: Not applying diff 0.20.1 -> 0.20.2 (sync in progress)
Sep 10 15:27:48 Cluster-Server-1 cib: [48709]: notice: cib_server_process_diff: Not applying diff 0.20.2 -> 0.20.3 (sync in progress)
Sep 10 15:27:48 Cluster-Server-1 cib: [48709]: notice: cib_server_process_diff: Not applying diff 0.20.3 -> 0.20.4 (sync in progress)
Sep 10 15:27:48 Cluster-Server-1 cib: [48709]: info: cib_replace_notify: Replaced: -1.-1.-1 -> 0.20.3 from Cluster-Server-2
Sep 10 15:27:48 Cluster-Server-1 crmd: [48715]: debug: do_cib_replaced: Updating the CIB after a replace: DC=false
Sep 10 15:27:48 Cluster-Server-1 attrd: [48713]: info: do_cib_replaced: Sending full refresh
Sep 10 15:27:48 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: master-p_Device_drive:0 (10000)
Sep 10 15:27:48 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:0'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[3])
Sep 10 15:27:48 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-master-p_Device_drive.0" name="master-p_Device_drive:0" value="10000" />
Sep 10 15:27:48 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: pingd (100)
Sep 10 15:27:48 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
Sep 10 15:27:48 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:27:48 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:27:48 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:27:48 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:27:48 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 190 for master-p_Device_drive:0=10000 passed
Sep 10 15:27:48 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 192 for probe_complete=true passed
Sep 10 15:27:48 Cluster-Server-1 lrmd: [48712]: debug: on_msg_add_rsc:client [48715] adds resource Target_iscsi3
Sep 10 15:27:48 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=16:26:7:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=Target_iscsi3_monitor_0
Sep 10 15:27:48 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 194 for pingd=100 passed
Sep 10 15:27:48 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc Target_iscsi3
Sep 10 15:27:48 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[52] on Target_iscsi3 for client 48715, its parameters: crm_feature_set=[3.0.6] iqn=[iqn.2005-07.com.example:vdisk.iscsi3] CRM_meta_timeout=[20000]  to the operation list.
Sep 10 15:27:48 Cluster-Server-1 lrmd: [48712]: info: rsc:Target_iscsi3 probe[52] (pid 61725)
Sep 10 15:27:48 Cluster-Server-1 lrmd: [48712]: debug: on_msg_add_rsc:client [48715] adds resource Lun_iscsi3
Sep 10 15:27:48 Cluster-Server-1 crmd: [48715]: debug: do_lrm_rsc_op: Performing key=17:26:7:81b7c738-e2a4-46c6-91bd-4df2c9c62d66 op=Lun_iscsi3_monitor_0
Sep 10 15:27:48 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op:2399: copying parameters for rsc Lun_iscsi3
Sep 10 15:27:48 Cluster-Server-1 lrmd: [48712]: debug: on_msg_perform_op: add an operation operation monitor[53] on Lun_iscsi3 for client 48715, its parameters: path=[/dev/drive-CSD/iscsi3_iSCSI] crm_feature_set=[3.0.6] lun=[0] handler=[vdisk_blockio] device_name=[iscsi3] CRM_meta_timeout=[20000] target_iqn=[iqn.2005-07.com.example:vdisk.iscsi3]  to the operation list.
Sep 10 15:27:48 Cluster-Server-1 lrmd: [48712]: info: rsc:Lun_iscsi3 probe[53] (pid 61726)
SCSTLun(Lun_iscsi3)[61726]:	2012/09/10_15:27:48 INFO: Lun_iscsi3 monitor : 7
SCSTTarget(Target_iscsi3)[61725]:	2012/09/10_15:27:48 DEBUG: Target_iscsi3 monitor : 7
Sep 10 15:27:48 Cluster-Server-1 lrmd: [48712]: WARN: Managed Target_iscsi3:monitor process 61725 exited with return code 7.
Sep 10 15:27:48 Cluster-Server-1 lrmd: [48712]: info: operation monitor[52] on Target_iscsi3 for client 48715: pid 61725 exited with return code 7
Sep 10 15:27:48 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce Target_iscsi3 after complete monitor op (interval=0)
Sep 10 15:27:48 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Target_iscsi3_monitor_0 (call=52, rc=7, cib-update=114, confirmed=true) not running
Sep 10 15:27:48 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'Target_iscsi3'
SCSTLun(Lun_iscsi3)[61726]:	2012/09/10_15:27:48 INFO: Lun_iscsi3 monitor : 7
Sep 10 15:27:48 Cluster-Server-1 lrmd: [48712]: WARN: Managed Lun_iscsi3:monitor process 61726 exited with return code 7.
Sep 10 15:27:48 Cluster-Server-1 lrmd: [48712]: info: operation monitor[53] on Lun_iscsi3 for client 48715: pid 61726 exited with return code 7
Sep 10 15:27:48 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: do_update_resource: Updating resouce Lun_iscsi3 after complete monitor op (interval=0)
Sep 10 15:27:48 Cluster-Server-1 crmd: [48715]: info: process_lrm_event: LRM operation Lun_iscsi3_monitor_0 (call=53, rc=7, cib-update=115, confirmed=true) not running
Sep 10 15:27:48 Cluster-Server-1 crmd: [48715]: debug: update_history_cache: Appending monitor op to history for 'Lun_iscsi3'
Sep 10 15:27:48 Cluster-Server-1 crmd: [48715]: debug: attrd_update_delegate: Sent update: probe_complete=true for localhost
Sep 10 15:27:48 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crmd: probe_complete=true
Sep 10 15:27:48 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: true, Current: true, Stored: true
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: rsc:FS_nfs2 monitor[43] (pid 61773)
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: info: operation monitor[43] on FS_nfs2 for client 48715: pid 61773 exited with return code 0
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: rsc:ExportFS_nfs2 monitor[45] (pid 61809)
exportfs(ExportFS_nfs2)[61809]:	2012/09/10_15:27:49 INFO: Directory /volumes/nfs2 is exported to * (started).
exportfs(ExportFS_nfs2)[61809]:	2012/09/10_15:27:49 INFO: Directory /volumes/nfs2 is exported to * (started).
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: info: operation monitor[45] on ExportFS_nfs2 for client 48715: pid 61809 exited with return code 0
Sep 10 15:27:49 Cluster-Server-1 cib: [48709]: debug: xmlfromIPC: Peer disconnected
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [61830] registered
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:61830] disconnected.
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:61830] is unregistered
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [61832] registered
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:61832] disconnected.
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:61832] is unregistered
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [61834] registered
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:61834] disconnected.
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:61834] is unregistered
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [61836] registered
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:61836] disconnected.
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:61836] is unregistered
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [61845] registered
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:61845] disconnected.
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:61845] is unregistered
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [61854] registered
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:61854] disconnected.
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:61854] is unregistered
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [61861] registered
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:61861] disconnected.
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:61861] is unregistered
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [61868] registered
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:61868] disconnected.
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:61868] is unregistered
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [61875] registered
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:61875] disconnected.
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:61875] is unregistered
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: on_msg_register:client lrmadmin [61883] registered
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: on_receive_cmd: the IPC to client [pid:61883] disconnected.
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: unregister_client: client lrmadmin [pid:61883] is unregistered
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_COUNT
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: crm_uptime: Current CPU usage is: 0s, 140000us
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: crm_compare_age: Loose: 140000 vs 590000 (usec)
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: info: do_election_count_vote: Election 14 (owner: Cluster-Server-2) lost: vote from Cluster-Server-2 (Uptime)
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: update_dc: Unset DC. Was Cluster-Server-2
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_ELECTION_CHECK
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: do_election_check: Ignore election check: we not in an election
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_PENDING: [ state=S_NOT_DC cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_LOG   
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: do_log: FSA: Input I_PENDING from do_election_count_vote() received in state S_NOT_DC
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: notice: do_state_transition: State transition S_NOT_DC -> S_PENDING [ input=I_PENDING cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_INTEGRATE_TIMER_STOP
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 61892)
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_FINALIZE_TIMER_STOP
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_START
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: crm_timer_start: Started Election Trigger (I_DC_TIMEOUT:20000ms), src=96
drbd(p_Device_drive:0)[61892]:	2012/09/10_15:27:49 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: handle_request: Raising I_JOIN_OFFER: join-12
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_JOIN_OFFER: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ]
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_CL_JOIN_REQUEST
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: info: update_dc: Set DC to Cluster-Server-2 (3.0.6)
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_offer_respond: do_cl_join_offer_respond added action A_DC_TIMER_STOP to the FSA
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: join_query_callback: Respond to join offer join-12
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: join_query_callback: Acknowledging Cluster-Server-2 as our DC
Sep 10 15:27:49 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:1'] does not exist
Sep 10 15:27:49 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update -22 for master-p_Device_drive:1=(null) passed
Sep 10 15:27:49 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:27:49 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:27:49 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:27:49 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:27:49 Cluster-Server-1 crm_attribute: [61925]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: handle_request: Raising I_JOIN_RESULT: join-12
Sep 10 15:27:49 Cluster-Server-1 cib: [48709]: debug: xmlfromIPC: Peer disconnected
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_JOIN_RESULT: [ state=S_PENDING cause=C_HA_MESSAGE origin=route_message ]
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_CL_JOIN_RESULT
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_finalize_respond: Confirming join join-12: join_ack_nack
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce FS_nfs2 after complete start op (interval=0)
Sep 10 15:27:49 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 197 for pingd=100 passed
Sep 10 15:27:49 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 199 for probe_complete=true passed
Sep 10 15:27:49 Cluster-Server-1 crm_attribute: [61925]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:27:49 Cluster-Server-1 crm_attribute: [61925]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce FS_nfs2 after complete monitor op (interval=20000)
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce ExportFS_nfs1 after complete monitor op (interval=0)
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce ExportFS_nfs2 after complete start op (interval=0)
Sep 10 15:27:49 Cluster-Server-1 crm_attribute: [61925]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:27:49 Cluster-Server-1 crm_attribute: [61925]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:27:49 Cluster-Server-1 crm_attribute: [61925]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:27:49 Cluster-Server-1 crm_attribute: [61925]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:27:49 Cluster-Server-1 crm_attribute: [61925]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:27:49 Cluster-Server-1 crm_attribute: [61925]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:27:49 Cluster-Server-1 crm_attribute: [61925]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:27:49 Cluster-Server-1 crm_attribute: [61925]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:27:49 Cluster-Server-1 crm_attribute: [61925]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:27:49 Cluster-Server-1 crm_attribute: [61925]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:27:49 Cluster-Server-1 crm_attribute: [61925]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:27:49 Cluster-Server-1 crm_attribute: [61925]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:27:49 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:27:49 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
Sep 10 15:27:49 Cluster-Server-1 cib: [48709]: debug: xmlfromIPC: Peer disconnected
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce ExportFS_nfs2 after complete monitor op (interval=10000)
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_NFS_Server:0 after complete start op (interval=0)
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_NFS_Server:0 after complete monitor op (interval=30000)
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce FS_nfs1 after complete monitor op (interval=0)
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce LVM_drive after complete monitor op (interval=0)
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Lun_iscsi1 after complete monitor op (interval=0)
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_PingD:0 after complete monitor op (interval=0)
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_PingD:0 after complete monitor op (interval=10000)
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Lun_iscsi2 after complete start op (interval=0)
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Lun_iscsi2 after complete monitor op (interval=10000)
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_Device_drive:0 after complete monitor op (interval=0)
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_Device_drive:0 after complete monitor op (interval=10000)
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_iSCSI_Daemon:0 after complete monitor op (interval=0)
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce p_iSCSI_Daemon:0 after complete monitor op (interval=30000)
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Target_iscsi2 after complete start op (interval=0)
drbd(p_Device_drive:0)[61892]:	2012/09/10_15:27:49 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[61892]:	2012/09/10_15:27:49 DEBUG: drive: Command output: 
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 61892 exited with return code 8
Sep 10 15:27:49 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Target_iscsi2 after complete monitor op (interval=10000)
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Target_iscsi3 after complete monitor op (interval=0)
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Target_iscsi1 after complete monitor op (interval=0)
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: create_operation_update: build_active_RAs: Updating resouce Lun_iscsi3 after complete monitor op (interval=0)
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: do_cl_join_finalize_respond: join-12: Join complete.  Sending local LRM status to Cluster-Server-2
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: attrd_update_delegate: Sent update: (null)=(null) for localhost
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: s_crmd_fsa: Processing I_NOT_DC: [ state=S_PENDING cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_LOG   
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: do_log: FSA: Input I_NOT_DC from do_cl_join_finalize_respond() received in state S_PENDING
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: notice: do_state_transition: State transition S_PENDING -> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_DC_TIMER_STOP
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_INTEGRATE_TIMER_STOP
Sep 10 15:27:49 Cluster-Server-1 crmd: [48715]: debug: do_fsa_action: actions:trace: 	// A_FINALIZE_TIMER_STOP
Sep 10 15:27:49 Cluster-Server-1 attrd: [48713]: notice: attrd_local_callback: Sending full refresh (origin=crmd)
Sep 10 15:27:49 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: master-p_Device_drive:0 (10000)
Sep 10 15:27:49 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:0'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[3])
Sep 10 15:27:49 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-master-p_Device_drive.0" name="master-p_Device_drive:0" value="10000" />
Sep 10 15:27:49 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: pingd (100)
Sep 10 15:27:49 Cluster-Server-1 attrd: [48713]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
Sep 10 15:27:49 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:27:49 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:27:49 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:27:49 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:27:49 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 201 for master-p_Device_drive:0=10000 passed
Sep 10 15:27:49 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 203 for probe_complete=true passed
Sep 10 15:27:49 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 205 for pingd=100 passed
Sep 10 15:27:49 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: cib_query: //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='master-p_Device_drive:1'] does not exist
Sep 10 15:27:49 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update -22 for master-p_Device_drive:1=(null) passed
Sep 10 15:27:49 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='pingd'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[2])
Sep 10 15:27:49 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-pingd" name="pingd" value="100" />
Sep 10 15:27:49 Cluster-Server-1 cib: [48709]: debug: cib_process_xpath: Processing cib_query op for //cib/status//node_state[@id='Cluster-Server-1']//transient_attributes//nvpair[@name='probe_complete'] (/cib/status/node_state[1]/transient_attributes/instance_attributes/nvpair[1])
Sep 10 15:27:49 Cluster-Server-1 attrd: [48713]: debug: find_nvpair_attr_delegate: Match <nvpair id="status-Cluster-Server-1-probe_complete" name="probe_complete" value="true" />
Sep 10 15:27:49 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 208 for pingd=100 passed
Sep 10 15:27:49 Cluster-Server-1 attrd: [48713]: debug: attrd_cib_callback: Update 210 for probe_complete=true passed
Sep 10 15:27:55 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi2 monitor[21] (pid 61980)
Sep 10 15:27:55 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi2 monitor[23] (pid 61981)
SCSTTarget(Target_iscsi2)[61980]:	2012/09/10_15:27:55 DEBUG: Target_iscsi2 monitor : 0
Sep 10 15:27:55 Cluster-Server-1 lrmd: [48712]: info: operation monitor[21] on Target_iscsi2 for client 48715: pid 61980 exited with return code 0
SCSTLun(Lun_iscsi2)[61981]:	2012/09/10_15:27:55 INFO: Lun_iscsi2 monitor : 0
SCSTLun(Lun_iscsi2)[61981]:	2012/09/10_15:27:55 INFO: Lun_iscsi2 monitor : 0
Sep 10 15:27:55 Cluster-Server-1 lrmd: [48712]: info: operation monitor[23] on Lun_iscsi2 for client 48715: pid 61981 exited with return code 0
Sep 10 15:27:58 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 62147)
Sep 10 15:27:59 Cluster-Server-1 lrmd: [48712]: debug: rsc:ExportFS_nfs2 monitor[45] (pid 62163)
exportfs(ExportFS_nfs2)[62163]:	2012/09/10_15:27:59 INFO: Directory /volumes/nfs2 is exported to * (started).
exportfs(ExportFS_nfs2)[62163]:	2012/09/10_15:27:59 INFO: Directory /volumes/nfs2 is exported to * (started).
Sep 10 15:27:59 Cluster-Server-1 lrmd: [48712]: info: operation monitor[45] on ExportFS_nfs2 for client 48715: pid 62163 exited with return code 0
Sep 10 15:27:59 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 62174)
drbd(p_Device_drive:0)[62174]:	2012/09/10_15:27:59 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:27:59 Cluster-Server-1 crm_attribute: [62204]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:27:59 Cluster-Server-1 crm_attribute: [62204]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:27:59 Cluster-Server-1 crm_attribute: [62204]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:27:59 Cluster-Server-1 crm_attribute: [62204]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:27:59 Cluster-Server-1 crm_attribute: [62204]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:27:59 Cluster-Server-1 crm_attribute: [62204]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:27:59 Cluster-Server-1 crm_attribute: [62204]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:27:59 Cluster-Server-1 crm_attribute: [62204]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:27:59 Cluster-Server-1 crm_attribute: [62204]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:27:59 Cluster-Server-1 crm_attribute: [62204]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:27:59 Cluster-Server-1 crm_attribute: [62204]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:27:59 Cluster-Server-1 crm_attribute: [62204]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:27:59 Cluster-Server-1 crm_attribute: [62204]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:27:59 Cluster-Server-1 crm_attribute: [62204]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:27:59 Cluster-Server-1 crm_attribute: [62204]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:27:59 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:27:59 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[62174]:	2012/09/10_15:27:59 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[62174]:	2012/09/10_15:27:59 DEBUG: drive: Command output: 
Sep 10 15:27:59 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:27:59 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 62174 exited with return code 8
Sep 10 15:28:00 Cluster-Server-1 attrd_updater: [62213]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:28:00 Cluster-Server-1 attrd_updater: [62213]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:28:00 Cluster-Server-1 attrd_updater: [62213]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:28:00 Cluster-Server-1 attrd_updater: [62213]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:28:00 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:28:00 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:28:00 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 62147 exited with return code 0
Sep 10 15:28:05 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi2 monitor[21] (pid 62215)
Sep 10 15:28:05 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi2 monitor[23] (pid 62216)
SCSTTarget(Target_iscsi2)[62215]:	2012/09/10_15:28:05 DEBUG: Target_iscsi2 monitor : 0
Sep 10 15:28:05 Cluster-Server-1 lrmd: [48712]: info: operation monitor[21] on Target_iscsi2 for client 48715: pid 62215 exited with return code 0
SCSTLun(Lun_iscsi2)[62216]:	2012/09/10_15:28:05 INFO: Lun_iscsi2 monitor : 0
SCSTLun(Lun_iscsi2)[62216]:	2012/09/10_15:28:05 INFO: Lun_iscsi2 monitor : 0
Sep 10 15:28:05 Cluster-Server-1 lrmd: [48712]: info: operation monitor[23] on Lun_iscsi2 for client 48715: pid 62216 exited with return code 0
Sep 10 15:28:09 Cluster-Server-1 lrmd: [48712]: debug: rsc:FS_nfs2 monitor[43] (pid 62278)
Sep 10 15:28:09 Cluster-Server-1 lrmd: [48712]: info: operation monitor[43] on FS_nfs2 for client 48715: pid 62278 exited with return code 0
Sep 10 15:28:09 Cluster-Server-1 lrmd: [48712]: debug: rsc:ExportFS_nfs2 monitor[45] (pid 62344)
exportfs(ExportFS_nfs2)[62344]:	2012/09/10_15:28:09 INFO: Directory /volumes/nfs2 is exported to * (started).
exportfs(ExportFS_nfs2)[62344]:	2012/09/10_15:28:09 INFO: Directory /volumes/nfs2 is exported to * (started).
Sep 10 15:28:09 Cluster-Server-1 lrmd: [48712]: info: operation monitor[45] on ExportFS_nfs2 for client 48715: pid 62344 exited with return code 0
Sep 10 15:28:09 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 62371)
drbd(p_Device_drive:0)[62371]:	2012/09/10_15:28:09 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:28:09 Cluster-Server-1 crm_attribute: [62401]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:28:09 Cluster-Server-1 crm_attribute: [62401]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:28:09 Cluster-Server-1 crm_attribute: [62401]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:28:09 Cluster-Server-1 crm_attribute: [62401]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:28:09 Cluster-Server-1 crm_attribute: [62401]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:28:09 Cluster-Server-1 crm_attribute: [62401]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:28:09 Cluster-Server-1 crm_attribute: [62401]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:28:09 Cluster-Server-1 crm_attribute: [62401]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:28:09 Cluster-Server-1 crm_attribute: [62401]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:28:09 Cluster-Server-1 crm_attribute: [62401]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:28:09 Cluster-Server-1 crm_attribute: [62401]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:28:09 Cluster-Server-1 crm_attribute: [62401]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:28:09 Cluster-Server-1 crm_attribute: [62401]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:28:09 Cluster-Server-1 crm_attribute: [62401]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:28:09 Cluster-Server-1 crm_attribute: [62401]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:28:09 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:28:09 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[62371]:	2012/09/10_15:28:09 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[62371]:	2012/09/10_15:28:09 DEBUG: drive: Command output: 
Sep 10 15:28:09 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:28:09 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 62371 exited with return code 8
Sep 10 15:28:10 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 62494)
Sep 10 15:28:12 Cluster-Server-1 attrd_updater: [62806]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:28:12 Cluster-Server-1 attrd_updater: [62806]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:28:12 Cluster-Server-1 attrd_updater: [62806]: debug: attrd_update_delegate: Sent update: pingd=100 for localhost
Sep 10 15:28:12 Cluster-Server-1 attrd_updater: [62806]: info: attrd_lazy_update: Updated pingd=100 for (null)
Sep 10 15:28:12 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from attrd_updater: pingd=100
Sep 10 15:28:12 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 100, Current: 100, Stored: 100
Sep 10 15:28:12 Cluster-Server-1 lrmd: [48712]: info: operation monitor[6] on p_PingD:0 for client 48715: pid 62494 exited with return code 0
Sep 10 15:28:15 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_NFS_Server:0 monitor[8] (pid 62873)
Sep 10 15:28:15 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_iSCSI_Daemon:0 monitor[5] (pid 62874)
Sep 10 15:28:15 Cluster-Server-1 lrmd: [48712]: info: operation monitor[8] on p_NFS_Server:0 for client 48715: pid 62873 exited with return code 0
Sep 10 15:28:15 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_NFS_Server:0:monitor:stdout) nfsd running

Sep 10 15:28:15 Cluster-Server-1 lrmd: [48712]: info: operation monitor[5] on p_iSCSI_Daemon:0 for client 48715: pid 62874 exited with return code 0
Sep 10 15:28:15 Cluster-Server-1 lrmd: [48712]: debug: rsc:Target_iscsi2 monitor[21] (pid 62879)
Sep 10 15:28:15 Cluster-Server-1 lrmd: [48712]: debug: rsc:Lun_iscsi2 monitor[23] (pid 62880)
SCSTLun(Lun_iscsi2)[62880]:	2012/09/10_15:28:16 INFO: Lun_iscsi2 monitor : 0
SCSTLun(Lun_iscsi2)[62880]:	2012/09/10_15:28:16 INFO: Lun_iscsi2 monitor : 0
Sep 10 15:28:16 Cluster-Server-1 lrmd: [48712]: info: operation monitor[23] on Lun_iscsi2 for client 48715: pid 62880 exited with return code 0
SCSTTarget(Target_iscsi2)[62879]:	2012/09/10_15:28:16 DEBUG: Target_iscsi2 monitor : 0
Sep 10 15:28:16 Cluster-Server-1 lrmd: [48712]: info: operation monitor[21] on Target_iscsi2 for client 48715: pid 62879 exited with return code 0
Sep 10 15:28:19 Cluster-Server-1 lrmd: [48712]: debug: rsc:ExportFS_nfs2 monitor[45] (pid 62953)
exportfs(ExportFS_nfs2)[62953]:	2012/09/10_15:28:19 INFO: Directory /volumes/nfs2 is exported to * (started).
exportfs(ExportFS_nfs2)[62953]:	2012/09/10_15:28:19 INFO: Directory /volumes/nfs2 is exported to * (started).
Sep 10 15:28:19 Cluster-Server-1 lrmd: [48712]: info: operation monitor[45] on ExportFS_nfs2 for client 48715: pid 62953 exited with return code 0
Sep 10 15:28:19 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_Device_drive:0 monitor[11] (pid 62965)
drbd(p_Device_drive:0)[62965]:	2012/09/10_15:28:20 DEBUG: drive: Calling /usr/sbin/crm_master -Q -l reboot -v 10000
Sep 10 15:28:20 Cluster-Server-1 crm_attribute: [62995]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_rw
Sep 10 15:28:20 Cluster-Server-1 crm_attribute: [62995]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/cib_callback
Sep 10 15:28:20 Cluster-Server-1 crm_attribute: [62995]: debug: cib_native_signon_raw: Connection to CIB successful
Sep 10 15:28:20 Cluster-Server-1 crm_attribute: [62995]: debug: query_node_uuid: Result section <nodes >
Sep 10 15:28:20 Cluster-Server-1 crm_attribute: [62995]: debug: query_node_uuid: Result section   <node id="Cluster-Server-1" type="normal" uname="Cluster-Server-1" />
Sep 10 15:28:20 Cluster-Server-1 crm_attribute: [62995]: debug: query_node_uuid: Result section   <node id="Cluster-Server-2" type="normal" uname="Cluster-Server-2" />
Sep 10 15:28:20 Cluster-Server-1 crm_attribute: [62995]: debug: query_node_uuid: Result section </nodes>
Sep 10 15:28:20 Cluster-Server-1 crm_attribute: [62995]: info: determine_host: Mapped Cluster-Server-1 to Cluster-Server-1
Sep 10 15:28:20 Cluster-Server-1 crm_attribute: [62995]: info: attrd_lazy_update: Connecting to cluster... 5 retries remaining
Sep 10 15:28:20 Cluster-Server-1 crm_attribute: [62995]: debug: init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Sep 10 15:28:20 Cluster-Server-1 crm_attribute: [62995]: debug: attrd_update_delegate: Sent update: master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:28:20 Cluster-Server-1 crm_attribute: [62995]: info: attrd_lazy_update: Updated master-p_Device_drive:0=10000 for Cluster-Server-1
Sep 10 15:28:20 Cluster-Server-1 crm_attribute: [62995]: info: main: Update master-p_Device_drive:0=10000 sent via attrd
Sep 10 15:28:20 Cluster-Server-1 crm_attribute: [62995]: debug: cib_native_signoff: Signing out of the CIB Service
Sep 10 15:28:20 Cluster-Server-1 crm_attribute: [62995]: info: crm_xml_cleanup: Cleaning up memory from libxml2
Sep 10 15:28:20 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: update message from crm_attribute: master-p_Device_drive:0=10000
Sep 10 15:28:20 Cluster-Server-1 attrd: [48713]: debug: attrd_local_callback: Supplied: 10000, Current: 10000, Stored: 10000
drbd(p_Device_drive:0)[62965]:	2012/09/10_15:28:20 DEBUG: drive: Exit code 0
drbd(p_Device_drive:0)[62965]:	2012/09/10_15:28:20 DEBUG: drive: Command output: 
Sep 10 15:28:20 Cluster-Server-1 lrmd: [48712]: debug: RA output: (p_Device_drive:0:monitor:stdout) 

Sep 10 15:28:20 Cluster-Server-1 lrmd: [48712]: info: operation monitor[11] on p_Device_drive:0 for client 48715: pid 62965 exited with return code 8
Sep 10 15:28:22 Cluster-Server-1 lrmd: [48712]: debug: rsc:p_PingD:0 monitor[6] (pid 63059)
