<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 TRANSITIONAL//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; CHARSET=UTF-8">
<META NAME="GENERATOR" CONTENT="GtkHTML/3.16.3">
</HEAD>
<BODY>
Hi Fabio,<BR>
<BR>
<BLOCKQUOTE TYPE=CITE>
<BLOCKQUOTE TYPE=CITE>
<FONT COLOR="#000000">There is an example on how to configure gfs2 also in the rhel6.5</FONT><BR>
<FONT COLOR="#000000">pacemaker documentation, using pcs.</FONT><BR>
</BLOCKQUOTE>
</BLOCKQUOTE>
<BR>
Super! Please share the link to this documentation. I only discovered the gfs2+pcs example with the rhel7 beta docs.<BR>
<BR>
Bob Haxo<BR>
<BR>
<BR>
<BR>
On Sat, 2014-01-04 at 16:56 +0100, Fabio M. Di Nitto wrote:
<BLOCKQUOTE TYPE=CITE>
<PRE>
<FONT COLOR="#000000">On 01/01/2014 01:57 AM, Bob Haxo wrote:</FONT>
<FONT COLOR="#000000">> Greetings ... Happy New Year!</FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> I am testing a configuration that is created from example in "Chapter 6.</FONT>
<FONT COLOR="#000000">> Configuring a GFS2 File System in a Cluster" of the "Red Hat Enterprise</FONT>
<FONT COLOR="#000000">> Linux 7.0 Beta Global File System 2" document. Only addition is</FONT>
<FONT COLOR="#000000">> stonith:fence_ipmilan. After encountering this issue when I configured</FONT>
<FONT COLOR="#000000">> with "crm", I re-configured using "pcs". I've included the configuration</FONT>
<FONT COLOR="#000000">> below.</FONT>
<FONT COLOR="#000000">Hold on a second here.. why are you using RHEL7 documentation to</FONT>
<FONT COLOR="#000000">configure RHEL6.5? Please don't mix :) there are some differences and we</FONT>
<FONT COLOR="#000000">definitely never tested mixing those up.</FONT>
<FONT COLOR="#000000">There is an example on how to configure gfs2 also in the rhel6.5</FONT>
<FONT COLOR="#000000">pacemaker documentation, using pcs.</FONT>
<FONT COLOR="#000000">I personally never saw this behaviour, so it's entirely possible that</FONT>
<FONT COLOR="#000000">mixing things up will result in unpredictable status.</FONT>
<FONT COLOR="#000000">Fabio</FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> I'm thinking that, in a 2-node cluster, if I run "stonith_admin -F</FONT>
<FONT COLOR="#000000">> <peer-node>", then <peer-node> should reboot and cleanly rejoin the</FONT>
<FONT COLOR="#000000">> cluster. This is not happening. </FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> What ultimately happens is that after the initially fenced node reboots,</FONT>
<FONT COLOR="#000000">> the system from which the stonith_admin -F command was run is fenced and</FONT>
<FONT COLOR="#000000">> reboots. The fencing stops there, leaving the cluster in an appropriate</FONT>
<FONT COLOR="#000000">> state.</FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> The issue seems to reside with clvmd/lvm. With the reboot of the</FONT>
<FONT COLOR="#000000">> initially fenced node, the clvmd resource fails on the surviving node,</FONT>
<FONT COLOR="#000000">> with a maximum of errors. I hypothesize there is an issue with locks,</FONT>
<FONT COLOR="#000000">> but have insufficient knowledge of clvmd/lvm locks to prove or disprove</FONT>
<FONT COLOR="#000000">> this hypothesis.</FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> Have I missed something ...</FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> 1) Is this expected behavior, and always the reboot of the fencing node</FONT>
<FONT COLOR="#000000">> happens?</FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> 2) Or, maybe I didn't correctly duplicate the Chapter 6 example?</FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> 3) Or, perhaps something is wrong or omitted from the Chapter 6 example?</FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> Suggestions will be much appreciated.</FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> Thanks,</FONT>
<FONT COLOR="#000000">> Bob Haxo</FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> RHEL6.5</FONT>
<FONT COLOR="#000000">> pacemaker-cli-1.1.10-14.el6_5.1.x86_64</FONT>
<FONT COLOR="#000000">> crmsh-1.2.5-55.1sgi709r3.rhel6.x86_64</FONT>
<FONT COLOR="#000000">> pacemaker-libs-1.1.10-14.el6_5.1.x86_64</FONT>
<FONT COLOR="#000000">> cman-3.0.12.1-59.el6_5.1.x86_64</FONT>
<FONT COLOR="#000000">> pacemaker-1.1.10-14.el6_5.1.x86_64</FONT>
<FONT COLOR="#000000">> corosynclib-1.4.1-17.el6.x86_64</FONT>
<FONT COLOR="#000000">> corosync-1.4.1-17.el6.x86_64</FONT>
<FONT COLOR="#000000">> pacemaker-cluster-libs-1.1.10-14.el6_5.1.x86_64</FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> Cluster Name: mici</FONT>
<FONT COLOR="#000000">> Corosync Nodes:</FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> Pacemaker Nodes:</FONT>
<FONT COLOR="#000000">> mici-admin mici-admin2</FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> Resources:</FONT>
<FONT COLOR="#000000">> Clone: clusterfs-clone</FONT>
<FONT COLOR="#000000">> Meta Attrs: interleave=true target-role=Started</FONT>
<FONT COLOR="#000000">> Resource: clusterfs (class=ocf provider=heartbeat type=Filesystem)</FONT>
<FONT COLOR="#000000">> Attributes: device=/dev/vgha2/lv_clust2 directory=/images fstype=gfs2</FONT>
<FONT COLOR="#000000">> options=defaults,noatime,nodiratime</FONT>
<FONT COLOR="#000000">> Operations: monitor on-fail=fence interval=30s</FONT>
<FONT COLOR="#000000">> (clusterfs-monitor-interval-30s)</FONT>
<FONT COLOR="#000000">> Clone: clvmd-clone</FONT>
<FONT COLOR="#000000">> Meta Attrs: interleave=true ordered=true target-role=Started</FONT>
<FONT COLOR="#000000">> Resource: clvmd (class=lsb type=clvmd)</FONT>
<FONT COLOR="#000000">> Operations: monitor on-fail=fence interval=30s</FONT>
<FONT COLOR="#000000">> (clvmd-monitor-interval-30s)</FONT>
<FONT COLOR="#000000">> Clone: dlm-clone</FONT>
<FONT COLOR="#000000">> Meta Attrs: interleave=true ordered=true</FONT>
<FONT COLOR="#000000">> Resource: dlm (class=ocf provider=pacemaker type=controld)</FONT>
<FONT COLOR="#000000">> Operations: monitor on-fail=fence interval=30s (dlm-monitor-interval-30s)</FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> Stonith Devices:</FONT>
<FONT COLOR="#000000">> Resource: p_ipmi_fencing_1 (class=stonith type=fence_ipmilan)</FONT>
<FONT COLOR="#000000">> Attributes: ipaddr=128.##.##.78 login=XXXXX passwd=XXXXX lanplus=1</FONT>
<FONT COLOR="#000000">> action=reboot pcmk_host_check=static-list pcmk_host_list=mici-admin</FONT>
<FONT COLOR="#000000">> Meta Attrs: target-role=Started</FONT>
<FONT COLOR="#000000">> Operations: monitor start-delay=30 interval=60s timeout=30</FONT>
<FONT COLOR="#000000">> (p_ipmi_fencing_1-monitor-60s)</FONT>
<FONT COLOR="#000000">> Resource: p_ipmi_fencing_2 (class=stonith type=fence_ipmilan)</FONT>
<FONT COLOR="#000000">> Attributes: ipaddr=128.##.##.220 login=XXXXX passwd=XXXXX lanplus=1</FONT>
<FONT COLOR="#000000">> action=reboot pcmk_host_check=static-list pcmk_host_list=mici-admin2</FONT>
<FONT COLOR="#000000">> Meta Attrs: target-role=Started</FONT>
<FONT COLOR="#000000">> Operations: monitor start-delay=30 interval=60s timeout=30</FONT>
<FONT COLOR="#000000">> (p_ipmi_fencing_2-monitor-60s)</FONT>
<FONT COLOR="#000000">> Fencing Levels:</FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> Location Constraints:</FONT>
<FONT COLOR="#000000">> Resource: p_ipmi_fencing_1</FONT>
<FONT COLOR="#000000">> Disabled on: mici-admin (score:-INFINITY)</FONT>
<FONT COLOR="#000000">> (id:location-p_ipmi_fencing_1-mici-admin--INFINITY)</FONT>
<FONT COLOR="#000000">> Resource: p_ipmi_fencing_2</FONT>
<FONT COLOR="#000000">> Disabled on: mici-admin2 (score:-INFINITY)</FONT>
<FONT COLOR="#000000">> (id:location-p_ipmi_fencing_2-mici-admin2--INFINITY)</FONT>
<FONT COLOR="#000000">> Ordering Constraints:</FONT>
<FONT COLOR="#000000">> start dlm-clone then start clvmd-clone (Mandatory)</FONT>
<FONT COLOR="#000000">> (id:order-dlm-clone-clvmd-clone-mandatory)</FONT>
<FONT COLOR="#000000">> start clvmd-clone then start clusterfs-clone (Mandatory)</FONT>
<FONT COLOR="#000000">> (id:order-clvmd-clone-clusterfs-clone-mandatory)</FONT>
<FONT COLOR="#000000">> Colocation Constraints:</FONT>
<FONT COLOR="#000000">> clusterfs-clone with clvmd-clone (INFINITY)</FONT>
<FONT COLOR="#000000">> (id:colocation-clusterfs-clone-clvmd-clone-INFINITY)</FONT>
<FONT COLOR="#000000">> clvmd-clone with dlm-clone (INFINITY)</FONT>
<FONT COLOR="#000000">> (id:colocation-clvmd-clone-dlm-clone-INFINITY)</FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> Cluster Properties:</FONT>
<FONT COLOR="#000000">> cluster-infrastructure: cman</FONT>
<FONT COLOR="#000000">> dc-version: 1.1.10-14.el6_5.1-368c726</FONT>
<FONT COLOR="#000000">> last-lrm-refresh: 1388530552</FONT>
<FONT COLOR="#000000">> no-quorum-policy: ignore</FONT>
<FONT COLOR="#000000">> stonith-enabled: true</FONT>
<FONT COLOR="#000000">> Node Attributes:</FONT>
<FONT COLOR="#000000">> mici-admin: standby=off</FONT>
<FONT COLOR="#000000">> mici-admin2: standby=off</FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> Last updated: Tue Dec 31 17:15:55 2013</FONT>
<FONT COLOR="#000000">> Last change: Tue Dec 31 16:57:37 2013 via cibadmin on mici-admin</FONT>
<FONT COLOR="#000000">> Stack: cman</FONT>
<FONT COLOR="#000000">> Current DC: mici-admin2 - partition with quorum</FONT>
<FONT COLOR="#000000">> Version: 1.1.10-14.el6_5.1-368c726</FONT>
<FONT COLOR="#000000">> 2 Nodes configured</FONT>
<FONT COLOR="#000000">> 8 Resources configured</FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> Online: [ mici-admin mici-admin2 ]</FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> Full list of resources:</FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> p_ipmi_fencing_1 (stonith:fence_ipmilan): Started mici-admin2</FONT>
<FONT COLOR="#000000">> p_ipmi_fencing_2 (stonith:fence_ipmilan): Started mici-admin</FONT>
<FONT COLOR="#000000">> Clone Set: clusterfs-clone [clusterfs]</FONT>
<FONT COLOR="#000000">> Started: [ mici-admin mici-admin2 ]</FONT>
<FONT COLOR="#000000">> Clone Set: clvmd-clone [clvmd]</FONT>
<FONT COLOR="#000000">> Started: [ mici-admin mici-admin2 ]</FONT>
<FONT COLOR="#000000">> Clone Set: dlm-clone [dlm]</FONT>
<FONT COLOR="#000000">> Started: [ mici-admin mici-admin2 ]</FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> Migration summary:</FONT>
<FONT COLOR="#000000">> * Node mici-admin:</FONT>
<FONT COLOR="#000000">> * Node mici-admin2:</FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> =====================================================</FONT>
<FONT COLOR="#000000">> crm_mon after the fenced node reboots. Shows the failure of clvmd that</FONT>
<FONT COLOR="#000000">> then</FONT>
<FONT COLOR="#000000">> occurs, which in turn triggers a fencing of that nnode</FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> Last updated: Tue Dec 31 17:06:55 2013</FONT>
<FONT COLOR="#000000">> Last change: Tue Dec 31 16:57:37 2013 via cibadmin on mici-admin</FONT>
<FONT COLOR="#000000">> Stack: cman</FONT>
<FONT COLOR="#000000">> Current DC: mici-admin - partition with quorum</FONT>
<FONT COLOR="#000000">> Version: 1.1.10-14.el6_5.1-368c726</FONT>
<FONT COLOR="#000000">> 2 Nodes configured</FONT>
<FONT COLOR="#000000">> 8 Resources configured</FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> Node mici-admin: UNCLEAN (online)</FONT>
<FONT COLOR="#000000">> Online: [ mici-admin2 ]</FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> Full list of resources:</FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> p_ipmi_fencing_1 (stonith:fence_ipmilan): Stopped</FONT>
<FONT COLOR="#000000">> p_ipmi_fencing_2 (stonith:fence_ipmilan): Started mici-admin</FONT>
<FONT COLOR="#000000">> Clone Set: clusterfs-clone [clusterfs]</FONT>
<FONT COLOR="#000000">> Started: [ mici-admin ]</FONT>
<FONT COLOR="#000000">> Stopped: [ mici-admin2 ]</FONT>
<FONT COLOR="#000000">> Clone Set: clvmd-clone [clvmd]</FONT>
<FONT COLOR="#000000">> clvmd (lsb:clvmd): FAILED mici-admin</FONT>
<FONT COLOR="#000000">> Stopped: [ mici-admin2 ]</FONT>
<FONT COLOR="#000000">> Clone Set: dlm-clone [dlm]</FONT>
<FONT COLOR="#000000">> Started: [ mici-admin mici-admin2 ]</FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> Migration summary:</FONT>
<FONT COLOR="#000000">> * Node mici-admin:</FONT>
<FONT COLOR="#000000">> clvmd: migration-threshold=1000000 fail-count=1 last-failure='Tue Dec</FONT>
<FONT COLOR="#000000">> 31 17:04:29 2013'</FONT>
<FONT COLOR="#000000">> * Node mici-admin2:</FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> Failed actions:</FONT>
<FONT COLOR="#000000">> clvmd_monitor_30000 on mici-admin 'unknown error' (1): call=60,</FONT>
<FONT COLOR="#000000">> status=Timed Out, la</FONT>
<FONT COLOR="#000000">> st-rc-change='Tue Dec 31 17:04:29 2013', queued=0ms, exec=0ms</FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> _______________________________________________</FONT>
<FONT COLOR="#000000">> Pacemaker mailing list: <A HREF="mailto:Pacemaker@oss.clusterlabs.org">Pacemaker@oss.clusterlabs.org</A></FONT>
<FONT COLOR="#000000">> <A HREF="http://oss.clusterlabs.org/mailman/listinfo/pacemaker">http://oss.clusterlabs.org/mailman/listinfo/pacemaker</A></FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">> Project Home: <A HREF="http://www.clusterlabs.org">http://www.clusterlabs.org</A></FONT>
<FONT COLOR="#000000">> Getting started: <A HREF="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</A></FONT>
<FONT COLOR="#000000">> Bugs: <A HREF="http://bugs.clusterlabs.org">http://bugs.clusterlabs.org</A></FONT>
<FONT COLOR="#000000">> </FONT>
<FONT COLOR="#000000">_______________________________________________</FONT>
<FONT COLOR="#000000">Pacemaker mailing list: <A HREF="mailto:Pacemaker@oss.clusterlabs.org">Pacemaker@oss.clusterlabs.org</A></FONT>
<FONT COLOR="#000000"><A HREF="http://oss.clusterlabs.org/mailman/listinfo/pacemaker">http://oss.clusterlabs.org/mailman/listinfo/pacemaker</A></FONT>
<FONT COLOR="#000000">Project Home: <A HREF="http://www.clusterlabs.org">http://www.clusterlabs.org</A></FONT>
<FONT COLOR="#000000">Getting started: <A HREF="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</A></FONT>
<FONT COLOR="#000000">Bugs: <A HREF="http://bugs.clusterlabs.org">http://bugs.clusterlabs.org</A></FONT>
</PRE>
</BLOCKQUOTE>
</BODY>
</HTML>