2010/12/11 Andrew Beekhof <span dir="ltr"><<a href="mailto:andrew@beekhof.net">andrew@beekhof.net</a>></span><br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div class="im">On Fri, Dec 10, 2010 at 4:59 PM, ruslan usifov <<a href="mailto:ruslan.usifov@gmail.com">ruslan.usifov@gmail.com</a>> wrote:<br>
> and to me what to do?<br>
<br>
</div>Nothing yet, there looks to be some memory corruption going on.<br>
With that file I've been able to reproduce locally. I'll let you know<br>
when there is a fix (hopefully very soon).<br>
<div><div></div><div class="h5"><br>
><br>
> 2010/12/10 Andrew Beekhof <<a href="mailto:andrew@beekhof.net">andrew@beekhof.net</a>><br>
>><br>
>> On Fri, Dec 10, 2010 at 11:16 AM, ruslan usifov <<a href="mailto:ruslan.usifov@gmail.com">ruslan.usifov@gmail.com</a>><br>
>> wrote:<br>
>> > you mean some think like this:<br>
>> ><br>
>> > Dec 07 15:14:05 storage1 crmd: [16003]: notice: save_cib_contents: Saved<br>
>> > CIB<br>
>> > contents after PE crash to /var/lib/pengine/pe-core<br>
>> > -121f59f1-ca5c-4ce4-996c-53f4aa617ac3.bz2<br>
>><br>
>> perfect<br>
>><br>
>> ><br>
>> ><br>
>> > ??<br>
>> > If so, i attached it to this email<br>
>> ><br>
>> ><br>
>> > 2010/12/10 Andrew Beekhof <<a href="mailto:andrew@beekhof.net">andrew@beekhof.net</a>><br>
>> >><br>
>> >> On Fri, Dec 10, 2010 at 10:18 AM, ruslan usifov<br>
>> >> <<a href="mailto:ruslan.usifov@gmail.com">ruslan.usifov@gmail.com</a>><br>
>> >> wrote:<br>
>> >> > I don't know how to see version of pacemaker, crm doesn't provide -v<br>
>> >> > (or<br>
>> >> > -V<br>
>> >> > or --version) option, but I got source from here<br>
>> >> > <a href="http://hg.clusterlabs.org/pacemaker/stable-1.0/archive/tip.tar.bz2" target="_blank">http://hg.clusterlabs.org/pacemaker/stable-1.0/archive/tip.tar.bz2</a>,<br>
>> >> > as<br>
>> >> > result I download Pacemaker-1-0-b0266dd5ffa9.tar.bz2<br>
>> >><br>
>> >> Ah, b0266dd5ffa9 tells me exactly what you're running.<br>
>> >><br>
>> >> I'm very concerned that can_run_resources() is crashing, looking at<br>
>> >> the code it should not be possible.<br>
>> >> Do you see any logs containing the text "save_cib_contents"? If so,<br>
>> >> it should mention a filename, could you attach it?<br>
>> >><br>
>> >> ><br>
>> >> ><br>
>> >> > and here is my backtrace:<br>
>> >> ><br>
>> >> > gdb /usr/lib/heartbeat/pengine /tmp/pengine.940.core<br>
>> >> > (gdb) bt<br>
>> >> > #0 0xb76879b8 in can_run_resources () from /usr/lib/libpengine.so.3<br>
>> >> > #1 0xb768dcd7 in node_list_attr_score () from<br>
>> >> > /usr/lib/libpengine.so.3<br>
>> >> > #2 0xb768de91 in ?? () from /usr/lib/libpengine.so.3<br>
>> >> > #3 0xb768e193 in rsc_merge_weights () from /usr/lib/libpengine.so.3<br>
>> >> > #4 0xb7681676 in native_merge_weights () from<br>
>> >> > /usr/lib/libpengine.so.3<br>
>> >> > #5 0xb7698654 in group_merge_weights () from<br>
>> >> > /usr/lib/libpengine.so.3<br>
>> >> > #6 0xb76a063a in ?? () from /usr/lib/libpengine.so.3<br>
>> >> > #7 0xb76a17ad in master_color () from /usr/lib/libpengine.so.3<br>
>> >> > #8 0xb7683f3e in stage5 () from /usr/lib/libpengine.so.3<br>
>> >> > #9 0xb76813aa in do_calculations () from /usr/lib/libpengine.so.3<br>
>> >> > #10 0xb7680991 in process_pe_message () from /usr/lib/libpengine.so.3<br>
>> >> > #11 0x08048ed5 in ?? ()<br>
>> >> > #12 0xb7611f4e in G_CH_dispatch_int () from /usr/lib/libplumb.so.2<br>
>> >> > #13 0xb727f5e5 in g_main_context_dispatch () from<br>
>> >> > /lib/libglib-2.0.so.0<br>
>> >> > #14 0xb72832d8 in ?? () from /lib/libglib-2.0.so.0<br>
>> >> > #15 0xb7283817 in g_main_loop_run () from /lib/libglib-2.0.so.0<br>
>> >> > #16 0x080494f8 in ?? ()<br>
>> >> > #17 0xb7337bd6 in __libc_start_main () from<br>
>> >> > /lib/tls/i686/cmov/libc.so.6<br>
>> >> > #18 0x08048dc1 in ?? ()<br>
>> >> ><br>
>> >> > _______________________________________________<br>
>> >> > Pacemaker mailing list: <a href="mailto:Pacemaker@oss.clusterlabs.org">Pacemaker@oss.clusterlabs.org</a><br>
>> >> > <a href="http://oss.clusterlabs.org/mailman/listinfo/pacemaker" target="_blank">http://oss.clusterlabs.org/mailman/listinfo/pacemaker</a><br>
>> >> ><br>
>> >> > Project Home: <a href="http://www.clusterlabs.org" target="_blank">http://www.clusterlabs.org</a><br>
>> >> > Getting started:<br>
>> >> > <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
>> >> > Bugs:<br>
>> >> ><br>
>> >> ><br>
>> >> > <a href="http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker" target="_blank">http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker</a><br>
>> >> ><br>
>> >> ><br>
>> >><br>
>> >> _______________________________________________<br>
>> >> Pacemaker mailing list: <a href="mailto:Pacemaker@oss.clusterlabs.org">Pacemaker@oss.clusterlabs.org</a><br>
>> >> <a href="http://oss.clusterlabs.org/mailman/listinfo/pacemaker" target="_blank">http://oss.clusterlabs.org/mailman/listinfo/pacemaker</a><br>
>> >><br>
>> >> Project Home: <a href="http://www.clusterlabs.org" target="_blank">http://www.clusterlabs.org</a><br>
>> >> Getting started:<br>
>> >> <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
>> >> Bugs:<br>
>> >><br>
>> >> <a href="http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker" target="_blank">http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker</a><br>
>> ><br>
>> ><br>
>> > _______________________________________________<br>
>> > Pacemaker mailing list: <a href="mailto:Pacemaker@oss.clusterlabs.org">Pacemaker@oss.clusterlabs.org</a><br>
>> > <a href="http://oss.clusterlabs.org/mailman/listinfo/pacemaker" target="_blank">http://oss.clusterlabs.org/mailman/listinfo/pacemaker</a><br>
>> ><br>
>> > Project Home: <a href="http://www.clusterlabs.org" target="_blank">http://www.clusterlabs.org</a><br>
>> > Getting started: <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
>> > Bugs:<br>
>> ><br>
>> > <a href="http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker" target="_blank">http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker</a><br>
>> ><br>
>> ><br>
>><br>
>> _______________________________________________<br>
>> Pacemaker mailing list: <a href="mailto:Pacemaker@oss.clusterlabs.org">Pacemaker@oss.clusterlabs.org</a><br>
>> <a href="http://oss.clusterlabs.org/mailman/listinfo/pacemaker" target="_blank">http://oss.clusterlabs.org/mailman/listinfo/pacemaker</a><br>
>><br>
>> Project Home: <a href="http://www.clusterlabs.org" target="_blank">http://www.clusterlabs.org</a><br>
>> Getting started: <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
>> Bugs:<br>
>> <a href="http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker" target="_blank">http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker</a><br>
><br>
><br>
> _______________________________________________<br>
> Pacemaker mailing list: <a href="mailto:Pacemaker@oss.clusterlabs.org">Pacemaker@oss.clusterlabs.org</a><br>
> <a href="http://oss.clusterlabs.org/mailman/listinfo/pacemaker" target="_blank">http://oss.clusterlabs.org/mailman/listinfo/pacemaker</a><br>
><br>
> Project Home: <a href="http://www.clusterlabs.org" target="_blank">http://www.clusterlabs.org</a><br>
> Getting started: <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
> Bugs:<br>
> <a href="http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker" target="_blank">http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker</a><br>
><br>
><br>
<br>
_______________________________________________<br>
Pacemaker mailing list: <a href="mailto:Pacemaker@oss.clusterlabs.org">Pacemaker@oss.clusterlabs.org</a><br>
<a href="http://oss.clusterlabs.org/mailman/listinfo/pacemaker" target="_blank">http://oss.clusterlabs.org/mailman/listinfo/pacemaker</a><br>
<br>
Project Home: <a href="http://www.clusterlabs.org" target="_blank">http://www.clusterlabs.org</a><br>
Getting started: <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" target="_blank">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a><br>
Bugs: <a href="http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker" target="_blank">http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker</a><br>
</div></div></blockquote></div><br><br>Thanks, but i have a question<br><br>I change my configuration in test inviroment (VMware) to follow:<br><br><div style="margin-left: 40px;">root@ubuntu:/home/ruslan# crm configure show<br>
node ubuntu<br>node ubuntu1<br>primitive drbd_web ocf:linbit:drbd \<br> params drbd_resource="web" \<br> op monitor interval="15s" \<br> op stop interval="0" timeout="300s" \<br>
op start interval="0" timeout="300s"<br>primitive iscsi_ip ocf:heartbeat:IPaddr2 \<br> params ip="192.168.83.5" nic="eth1:1" cidr_netmask="24" \<br> op monitor interval="10s" \<br>
meta target-role="Started"<br>primitive iscsi_lun1_web_target ocf:heartbeat:iSCSILogicalUnit \<br> params lun="1" path="/dev/drbd0" target_iqn="iqn.2010-06.playrix.local:san.web" implementation="iet"<br>
primitive iscsi_target lsb:iscsitarget<br>primitive iscsi_web_target ocf:heartbeat:iSCSITarget \<br> params iqn="iqn.2010-06.playrix.local:san.web" implementation="iet" \<br> op monitor interval="10s" timeout="30s" depth="0" \<br>
meta target-role="Started"<br>group iscsi iscsi_ip iscsi_target iscsi_web_target iscsi_lun1_web_target \<br> meta target-role="Started"<br>ms ms_drbd_web drbd_web \<br> meta master-max="1" master-node-max="1" clone-max="2" clone-node-max="1" notify="true" target-role="Started" is-managed="true"<br>
colocation ms_drbd_with_iscsi inf: iscsi ms_drbd_web:Master<br>order o1 0: ms_drbd_web:promote iscsi_ip<br>order o2 inf: iscsi_ip iscsi_target:start<br>order o3 inf: iscsi_target iscsi_web_target<br>order o4 inf: iscsi_web_target iscsi_lun1_web_target<br>
property $id="cib-bootstrap-options" \<br> dc-version="1.0.10-b0266dd5ffa9c51377c68b1f29d6bc84367f51dd" \<br> cluster-infrastructure="openais" \<br> expected-quorum-votes="2" \<br>
stonith-enabled="false" \<br> no-quorum-policy="ignore" \<br> last-lrm-refresh="1292028155"<br>rsc_defaults $id="rsc-options" \<br> resource-stickness="100"<br>
<br><br></div>and when start one node (without second) i got follow output:<br><br><div style="margin-left: 40px;">root@ubuntu:/home/ruslan# crm_mon -1<br>Stack: openais<br>Current DC: ubuntu - partition WITHOUT quorum<br>
Version: 1.0.10-b0266dd5ffa9c51377c68b1f29d6bc84367f51dd<br>2 Nodes configured, 2 expected votes<br>2 Resources configured.<br>============<br><br>Online: [ ubuntu ]<br>OFFLINE: [ ubuntu1 ]<br><br> Master/Slave Set: ms_drbd_web<br>
Slaves: [ ubuntu ]<br> Stopped: [ drbd_web:1 ]<br><br>Failed actions:<br> iscsi_target_monitor_0 (node=ubuntu, call=4, rc=1, status=complete): unknown error<br> iscsi_web_target_monitor_0 (node=ubuntu, call=5, rc=5, status=complete): not installed<br>
iscsi_lun1_web_target_monitor_0 (node=ubuntu, call=6, rc=5, status=complete): not installed<br><br></div><div style="margin-left: 40px;"><br></div>Hm it very strange. As i expected this rules:<br><br> order o1 inf: ms_drbd_web:promote iscsi_ip<br>
<div style="margin-left: 40px;">order o2 inf: iscsi_ip iscsi_target:start<br>
order o3 inf: iscsi_target iscsi_web_target<br>
order o4 inf: iscsi_web_target iscsi_lun1_web_target<br></div>
<br>prevent launch and monitor of all resource that depends on drbd, but with Failed actions I confused, why they monitored?<br><br>