<div dir="ltr">Greetings,<br><br>Heh. Well, the comment in corosync.conf makes sense to me now.<br>Thanks, I've fixed that.<br><br><br>Here's my corosync.conf<br>----------------------------------------<br>totem {<br> version: 2<br><br> crypto_cipher: none<br> crypto_hash: none<br><br> interface {<br> ringnumber: 0<br> bindnetaddr: 10.1.0.0<br> mcastaddr: 239.255.1.1<br> mcastport: 5405<br> ttl: 1<br> }<br> cluster_name: pecan<br>}<br><br>logging {<br> fileline: off<br> to_stderr: no<br> to_logfile: yes<br> logfile: /var/log/cluster/corosync.log<br> to_syslog: yes<br> debug: off<br> timestamp: on<br> logger_subsys {<br> subsys: QUORUM<br> debug: off<br> }<br>}<br><br>quorum {<br> provider: corosync_votequorum<br> two_node: 1<br> wait_for_all: 1<br>}<br>service {<br> name: pacemaker<br> ver: 1<br>}<br>nodelist {<br> node {<br> ring0_addr: smoking<br> nodeid: 1<br> }<br> node {<br> ring0_addr: mars<br> nodeid: 2<br> }<br>}<br>----------------------------------------<br><br>And a few things are behaving better than they did before.<br><br>At the moment my goal is to set up a partition as drbd.<br>In the interest of bandwidth I will show the commands that<br>I use and the result I finally get.<br><br>----------------------------------------<br>pcs cluster auth smoking mars<br>pcs property set stonith-enabled=true<br>stonith_admin --metadata --agent fence_pcmk<br>cibadmin -C -o resources --xml-file stonith.xml<br>pcs resource create floating_ip IPaddr2 ip=10.1.2.101 cidr_netmask=32<br>pcs resource defaults resource-stickiness=100<br>----------------------------------------<br><br>And at this point, all appears well. My pcs status output looks like<br>I think it should.<br><br>Now, of course, I admit that setting up the floating_ip is<br>not relevant to my goal of a drbd backed filesystem, but I've been<br>doing it as a sanity check.<br><br>On to drbd<br>----------------------------------------<br>modprobe drbd<br>systemctl start drbd.service<br>[root@smoking cluster]# cat /proc/drbd<br>version: 8.4.8-1 (api:1/proto:86-101)<br>GIT-hash: 22b4c802192646e433d3f7399d578ec7fecc6272 build by mockbuild@, 2016-10-<br>13 19:58:26<br> 0: cs:Connected ro:Secondary/Secondary ds:Diskless/Diskless C r-----<br> ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0<br> 1: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r-----<br> ns:0 nr:10574 dw:10574 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0<br> 2: cs:Connected ro:Secondary/Secondary ds:Diskless/Diskless C r-----<br> ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0<br>----------------------------------------<br>Again, this is stuff that hung around from the previous incarnation.<br>But it looks okay to me. I'm planning to use the '1' device.<br>The above is run on the secondary machine, so Secondary/Primary is<br>correct. And UpToDate/UpToDate looks right to me.<br><br>Now it goes south. The mkfs.xfs appears to work, but that's not<br>relevant anyway, right?<br>----------------------------------------<br>pcs resource create BravoSpace \<br> ocf:linbit:drbd drbd_resource=bravo \<br> op monitor interval=60s<br><br>[root@smoking ~]# pcs status<br>Cluster name: pecan<br>Last updated: Sat Oct 15 01:33:37 2016 Last change: Sat Oct 15 01:18:56<br> 2016 by root via cibadmin on mars<br>Stack: corosync<br>Current DC: mars (version 1.1.13-10.el7_2.4-44eb2dd) - partition with quorum<br>2 nodes and 3 resources configured<br><br>Node mars: UNCLEAN (online)<br>Node smoking: UNCLEAN (online)<br><br>Full list of resources:<br><br> Fencing (stonith:fence_pcmk): Started mars<br> floating_ip (ocf::heartbeat:IPaddr2): Started mars<br> BravoSpace (ocf::linbit:drbd): FAILED[ smoking mars ]<br><br>Failed Actions:<br>* BravoSpace_stop_0 on smoking 'not configured' (6): call=18, status=complete, e<br>xitreason='none',<br> last-rc-change='Sat Oct 15 01:18:56 2016', queued=0ms, exec=63ms<br>* BravoSpace_stop_0 on mars 'not configured' (6): call=18, status=complete, exit<br>reason='none',<br> last-rc-change='Sat Oct 15 01:18:56 2016', queued=0ms, exec=60ms<br><br><br>PCSD Status:<br> smoking: Online<br> mars: Online<br><br>Daemon Status:<br> corosync: active/disabled<br> pacemaker: active/disabled<br> pcsd: active/disabled<br>----------------------------------------<br>I've looked in /var/log/cluster/corosync.log and it doesn't seem<br>happy but I don't know what I'm looking at. On the primary<br>machine it's 1800+ lines on the secondary it's 600+ lines.<br>There are 337 lines just with BravoSpace in them.<br>One of them says<br>drbd(BravoSpace)[3295]: 2016/10/15_01:18:56 ERROR: meta parameter misconfigured,<br> expected clone-max -le 2, but found unset.<br>But I tried adding clone-max=2 but the command barfed-- that's not a legal<br>parameter.<br><br>So, what's wrong? (I'm a newbie, of course.)<br><br>I did a pcs resource cleanup . That shut down fencing and the IP.<br>I tried pcs cluster start to get them back, no help. <br>I did pcs cluster standby smoking, and then unstandby smoking.<br>The ip started, but fencing has failed on BOTH machines.<br>I can't see what I'm doing wrong.<br><br>Thanks. I realize I'm consuming your time on the cheap.<br><br></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Oct 14, 2016 at 3:33 PM, Dimitri Maziuk <span dir="ltr"><<a href="mailto:dmaziuk@bmrb.wisc.edu" target="_blank">dmaziuk@bmrb.wisc.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On 10/14/2016 02:48 PM, Jay Scott wrote:<br>
<br>
</span><span class="">> When I "start over" I stop all the services, delete the packages,<br>
> empty the configs and logs as best I know how. But this doesn't<br>
> completely clear everything: the drbd metadata is evidently still<br>
> on the partitions I've set aside for it.<br>
<br>
</span>If it's small enough, dd if=/dev/zero of=/your/partition<br>
<br>
Get DRBD working and fully sync'ed outside of the cluster before you<br>
start adding it.<br>
<span class="HOEnZb"><font color="#888888"><br>
--<br>
Dimitri Maziuk<br>
Programmer/sysadmin<br>
BioMagResBank, UW-Madison -- <a href="http://www.bmrb.wisc.edu" rel="noreferrer" target="_blank">http://www.bmrb.wisc.edu</a><br>
<br>
</font></span><br>______________________________<wbr>_________________<br>
Users mailing list: <a href="mailto:Users@clusterlabs.org">Users@clusterlabs.org</a><br>
<a href="http://clusterlabs.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://clusterlabs.org/<wbr>mailman/listinfo/users</a><br>
<br>
Project Home: <a href="http://www.clusterlabs.org" rel="noreferrer" target="_blank">http://www.clusterlabs.org</a><br>
Getting started: <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" rel="noreferrer" target="_blank">http://www.clusterlabs.org/<wbr>doc/Cluster_from_Scratch.pdf</a><br>
Bugs: <a href="http://bugs.clusterlabs.org" rel="noreferrer" target="_blank">http://bugs.clusterlabs.org</a><br>
<br></blockquote></div><br></div>