<div dir="ltr">Hi,<div><br></div><div>I'm puppetizing resource deployment for pacemaker and corosync, and as part of it, am creating a resource on one of three nodes of a cluster. The problem is that I'm seeing RecurringOp errors during resource creation, which are probably not allowing failover a resource. The resource creation seems to go through fine, but these recurringOp errors always result after resource creation (I'm pasting outputs of two different commands below):</div>
<div><br></div><div><br></div><div><span style="font-family:Menlo;font-size:11px">***************************</span></div><div><p style="margin:0px;font-size:11px;font-family:Menlo">vagrant@precise64b:/vagrant/puppet-environments/modules/f5_lbaas/tests$ sudo crm status</p>

<p style="margin:0px;font-size:11px;font-family:Menlo">============</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">Last updated: Wed Jul  2 03:52:30 2014</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">Last change: Wed Jul  2 03:38:20 2014 via cibadmin on precise64b</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">Stack: cman</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">Current DC: precise64b - partition with quorum</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">Version: 1.1.6-9971ebba4494012a93c03b40a2c58ec0eb60f50c</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">3 Nodes configured, unknown expected votes</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">3 Resources configured.</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">============</p>
<p style="margin:0px;font-size:11px;font-family:Menlo;min-height:13px"><br></p>
<p style="margin:0px;font-size:11px;font-family:Menlo">Online: [ precise64b precise64c precise64a ]</p>
<p style="margin:0px;font-size:11px;font-family:Menlo;min-height:13px"><br></p>
<p style="margin:0px;font-size:11px;font-family:Menlo"> f5-lbaas-agent-10.6.143.121_resource<span class="" style="white-space:pre">  </span>(lsb:f5-lbaas-agent-10.6.143.121):<span class="" style="white-space:pre">        </span>Started precise64c</p>

<p style="margin:0px;font-size:11px;font-family:Menlo"> f5-lbaas-agent-10.6.143.122_resource<span class="" style="white-space:pre">  </span>(lsb:f5-lbaas-agent-10.6.143.122):<span class="" style="white-space:pre">        </span>Started precise64b</p>

<p style="margin:0px;font-size:11px;font-family:Menlo"> f5-lbaas-agent-10.6.143.123_resource<span class="" style="white-space:pre">  </span>(lsb:f5-lbaas-agent-10.6.143.123):<span class="" style="white-space:pre">        </span>Started precise64b</p>

<p style="margin:0px;font-size:11px;font-family:Menlo;min-height:13px"><br></p>
<p style="margin:0px;font-size:11px;font-family:Menlo">Failed actions:</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">    f5-lbaas-agent-10.6.143.120_resource_monitor_0 (node=precise64b, call=2, rc=5, status=complete): not installed</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">    f5-lbaas-agent-10.6.143.121_resource_monitor_0 (node=precise64b, call=3, rc=5, status=complete): not installed</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">    f5-lbaas-agent-10.6.143.122_resource_monitor_0 (node=precise64c, call=7, rc=5, status=complete): not installed</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">    f5-lbaas-agent-10.6.143.123_resource_monitor_0 (node=precise64c, call=8, rc=5, status=complete): not installed</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">    f5-lbaas-agent-10.6.143.120_resource_monitor_0 (node=precise64a, call=2, rc=5, status=complete): not installed</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">    f5-lbaas-agent-10.6.143.121_resource_monitor_0 (node=precise64a, call=3, rc=5, status=complete): not installed</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">    f5-lbaas-agent-10.6.143.122_resource_monitor_0 (node=precise64a, call=4, rc=5, status=complete): not installed</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">    f5-lbaas-agent-10.6.143.123_resource_monitor_0 (node=precise64a, call=5, rc=5, status=complete): not installed</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">vagrant@precise64b:/vagrant/puppet-environments/modules/f5_lbaas/tests$ </p><p style="margin:0px;font-size:11px;font-family:Menlo"><br></p><p style="margin:0px;font-size:11px;font-family:Menlo">
<br></p><p style="margin:0px;font-size:11px;font-family:Menlo">***************************</p></div><div><br></div><div><p style="margin:0px;font-size:11px;font-family:Menlo">vagrant@precise64b:/vagrant/puppet-environments/modules/f5_lbaas/tests$ sudo crm_verify -L -V</p>

<p style="margin:0px;font-size:11px;font-family:Menlo">crm_verify[15183]: 2014/07/02_03:39:13 ERROR: RecurringOp: Invalid recurring action f5-lbaas-agent-10.6.143.121_resource-start-10 wth name: 'start'</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">crm_verify[15183]: 2014/07/02_03:39:13 ERROR: RecurringOp: Invalid recurring action f5-lbaas-agent-10.6.143.121_resource-stop-10 wth name: 'stop'</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">crm_verify[15183]: 2014/07/02_03:39:13 ERROR: RecurringOp: Invalid recurring action f5-lbaas-agent-10.6.143.122_resource-start-10 wth name: 'start'</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">crm_verify[15183]: 2014/07/02_03:39:13 ERROR: RecurringOp: Invalid recurring action f5-lbaas-agent-10.6.143.122_resource-stop-10 wth name: 'stop'</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">Errors found during check: config not valid</p>
<p style="margin:0px;font-size:11px;font-family:Menlo">vagrant@precise64b:/vagrant/puppet-environments/modules/f5_lbaas/tests$</p><p style="margin:0px;font-size:11px;font-family:Menlo">***************************<br></p>
<p style="margin:0px;font-size:11px;font-family:Menlo"><br></p><p style="margin:0px;font-size:11px;font-family:Menlo"><br></p><p style="margin:0px">What do these errors signify? I found one email exchange on a pacemaker ML that suggested that we shouldn't be using start intervals and timeouts, and same with stop, since that would mean that pacemaker would attempt to restart the resource every x seconds, timeout every y seconds, and repeat that. (Link: <a href="http://lists.linbit.com/pipermail/drbd-user/2011-September/016938.html">http://lists.linbit.com/pipermail/drbd-user/2011-September/016938.html</a>)<br>
</p><p style="margin:0px"><br></p><p style="margin:0px">My understanding was that the start interval would apply in case of restart attempts upon detection of a resource as being down. Nevertheless, I removed these parameters and created a third resource (the first two, I created with these parameters), and I still see the same monitor related errors for the third resource (<span style="font-family:Menlo;font-size:11px">f5-lbaas-agent-10.6.143.123_resource_monitor_0) </span>in the sudo crm status command output. I don't however understand why this resource doesn't show up in the crm_verify -L -V output.</p>
<p style="margin:0px"><br></p><p style="margin:0px">Here are the two CLIs I use to create the resources:</p><p style="margin:0px"><br></p><p style="margin:0px;font-size:11px;font-family:Menlo">sudo crm configure primitive $pmk_res_name $pmk_cont_type:$service_name op monitor interval="$mon_interval" timeout="$mon_timeout" op start interval="$start_interval" timeout="$start_timeout" op stop interval="$stop_interval" timeout="$stop_timeout</p>
<p style="margin:0px;font-size:11px;font-family:Menlo"><br></p><p style="margin:0px;font-size:11px;font-family:Menlo"><br></p><p style="margin:0px;font-size:11px;font-family:Menlo">sudo crm configure primitive $pmk_res_name $pmk_cont_type:$service_name op monitor interval="$mon_interval" timeout="$mon_timeout"</p>
<p style="margin:0px"><br></p><p style="margin:0px"><br></p><p style="margin:0px">The bottom-line is that if I halt the VM running any of these resources, the resource isn't failing over to another VM. I'm not sure what the exact cause is - any help would be greatly appreciated!</p>
<p style="margin:0px"><br></p><p style="margin:0px"><br></p><p style="margin:0px">Thanks,</p><p style="margin:0px">Regards,</p><p style="margin:0px">Vijay</p></div></div>