Diff hostcache... OK
Diff members.txt... OK
Diff cib.xml... OK
Diff crm_mon.txt... OK
Diff ha.cf... OK
Diff logd.cf... no /tmp/hb_report/*/logd.cf :/
Diff sysinfo.txt... --- /tmp/hb_report/bl460g1n13/sysinfo.txt	2011-09-13 21:54:58.000000000 +0900
+++ /tmp/hb_report/bl460g1n14/sysinfo.txt	2011-09-13 21:54:58.000000000 +0900
@@ -1,5 +1,5 @@
 heartbeat version: 3.0.5 (node: 7e3a82377fa8c88b4d9ee47e29020d4531f4629a)
-cluster-glue: 1.0.7 (ea850bf2e35e+ default tip)
+cluster-glue: 1.0.7 (5e06b2ddd24b37ad6c1c25d958d7a9dda7d02f93)
 resource-agents: # Build version: $Format:%H$
 CRM Version: 1.0.11 (1554a83db0d3c3e546cfd3aaff6af1184f79ee87)
 pacemaker 1.0.11-1.2.el5 - (none) x86_64
@@ -14,56 +14,6 @@
 package libcorosync4 is not installed
 resource-agents 3.9.1-1.1.el5 - (none) x86_64
 cluster-glue 1.0.7-1.1.el5 - (none) x86_64
-.......T    /etc/init.d/logd
-S.5....T    /usr/lib64/heartbeat/ha_logd
-S.5....T    /usr/lib64/heartbeat/lrmd
-S.5....T    /usr/lib64/heartbeat/plugins/InterfaceMgr/generic.so
-S.5....T    /usr/lib64/heartbeat/plugins/RAExec/heartbeat.so
-S.5....T    /usr/lib64/heartbeat/plugins/RAExec/lsb.so
-S.5....T    /usr/lib64/heartbeat/plugins/RAExec/ocf.so
-.......T    /usr/lib64/stonith/plugins/external/drac5
-.......T    /usr/lib64/stonith/plugins/external/dracmc-telnet
-.......T    /usr/lib64/stonith/plugins/external/hmchttp
-.......T    /usr/lib64/stonith/plugins/external/ibmrsa
-.......T    /usr/lib64/stonith/plugins/external/ibmrsa-telnet
-S.5....T    /usr/lib64/stonith/plugins/external/ipmi
-.......T    /usr/lib64/stonith/plugins/external/ippower9258
-.......T    /usr/lib64/stonith/plugins/external/kdumpcheck
-.......T    /usr/lib64/stonith/plugins/external/nut
-S.5....T    /usr/lib64/stonith/plugins/external/rackpdu
-.......T    /usr/lib64/stonith/plugins/external/riloe
-S.5....T    /usr/lib64/stonith/plugins/external/sbd
-.......T    /usr/lib64/stonith/plugins/external/vmware
-.......T    /usr/lib64/stonith/plugins/external/xen0
-.......T    /usr/lib64/stonith/plugins/external/xen0-ha
-S.5....T    /usr/lib64/stonith/plugins/stonith2/apcmaster.so
-S.5....T    /usr/lib64/stonith/plugins/stonith2/apcsmart.so
-S.5....T    /usr/lib64/stonith/plugins/stonith2/baytech.so
-S.5....T    /usr/lib64/stonith/plugins/stonith2/bladehpi.so
-S.5....T    /usr/lib64/stonith/plugins/stonith2/cyclades.so
-S.5....T    /usr/lib64/stonith/plugins/stonith2/drac3.so
-S.5....T    /usr/lib64/stonith/plugins/stonith2/external.so
-S.5....T    /usr/lib64/stonith/plugins/stonith2/ibmhmc.so
-S.5....T    /usr/lib64/stonith/plugins/stonith2/meatware.so
-S.5....T    /usr/lib64/stonith/plugins/stonith2/nw_rpc100s.so
-S.5....T    /usr/lib64/stonith/plugins/stonith2/rcd_serial.so
-S.5....T    /usr/lib64/stonith/plugins/stonith2/rhcs.so
-.......T    /usr/lib64/stonith/plugins/stonith2/ribcl.py
-S.5....T    /usr/lib64/stonith/plugins/stonith2/rps10.so
-S.5....T    /usr/lib64/stonith/plugins/stonith2/suicide.so
-S.5....T    /usr/lib64/stonith/plugins/stonith2/wti_nps.so
-.......T    /usr/lib64/stonith/plugins/xen0-ha-dom0-stonith-helper
-S.5....T    /usr/sbin/ha_logger
-S.5....T    /usr/sbin/hb_report
-S.5....T    /usr/sbin/lrmadmin
-S.5....T    /usr/sbin/meatclient
-S.5....T    /usr/sbin/sbd
-S.5....T    /usr/sbin/stonith
-.......T    /usr/share/cluster-glue/combine-logs.pl
-.......T    /usr/share/cluster-glue/ha_cf_support.sh
-.......T    /usr/share/cluster-glue/ha_log.sh
-.......T    /usr/share/cluster-glue/openais_conf_support.sh
-S.5....T    /usr/share/cluster-glue/utillib.sh
 package libglue2 is not installed
 package ldirectord is not installed
 heartbeat 3.0.5-1.1.el5 - (none) x86_64

WARN: crm_verify reported warnings at bl460g1n13:
crm_verify[15167]: 2011/09/13_21:54:55 WARN: unpack_nodes: Blind faith: not fencing unseen nodes
crm_verify[15167]: 2011/09/13_21:54:55 WARN: unpack_rsc_op: Processing failed op dummy01_monitor_10000 on bl460g1n13: not running (7)
WARN: crm_verify reported warnings at bl460g1n14:
crm_verify[3893]: 2011/09/13_21:54:55 WARN: unpack_nodes: Blind faith: not fencing unseen nodes
crm_verify[3893]: 2011/09/13_21:54:55 WARN: unpack_rsc_op: Processing failed op dummy01_monitor_10000 on bl460g1n13: not running (7)
WARN: problem with permissions/ownership at bl460g1n13:
/var/lib/heartbeat/ccm wrong type or doesn't exist
WARN: problem with permissions/ownership at bl460g1n14:
/var/lib/heartbeat/ccm wrong type or doesn't exist

Log patterns:
Sep 13 21:46:25 bl460g1n13 drbd[9572]: ERROR: remote  dstate is DUnknown
Sep 13 21:46:28 bl460g1n13 drbd[9613]: ERROR: remote  dstate is DUnknown
Sep 13 21:46:28 bl460g1n13 drbd[9613]: ERROR: remote  dstate is DUnknown
Sep 13 21:46:25 bl460g1n13 drbd[9572]: ERROR: remote  dstate is DUnknown
Sep 13 21:46:28 bl460g1n13 drbd[9613]: ERROR: remote  dstate is DUnknown
Sep 13 21:46:28 bl460g1n13 drbd[9613]: ERROR: remote  dstate is DUnknown
