[ClusterLabs] 'pcs stonith update' takes, then reverts

Digimer lists at alteeve.ca
Fri Jul 23 21:46:45 EDT 2021


After a LOT of hassle, I finally got it updated, but OMG it was painful.

I degraded the cluster (unsure if needed), set maintenance mode, deleted
the stonith levels, deleted the stonith devices, recreated them with the
updated values, recreated the stonith levels, and finally disabled
maintenance mode.

It should not have been this hard, right? Why is heck would it be that
pacemaker kept "rolling back" to old configs? I'd delete the stonith
levels, delete one PDU stonith resource, delete a second, and suddenly
all the levels were back and the resources I had just removed came back
with the old configs.

Certainly I was doing something wrong, but what?

digimer

On 2021-07-23 8:04 p.m., Digimer wrote:
> Update;
>
>   Appears I can't even delete the damn things. They re-appeared after
> doing a [pcs stonith remove <id>'!.
>
>   Wow.
>
> digimer
>
> On 2021-07-23 7:56 p.m., Digimer wrote:
>> Hi all,
>>
>>   Got a really odd one here...
>>
>>   I had a cluster in the lab where it was built and tested. Then we
>> deployed it, and I've been trying to update the stonith config. It
>> _seems_ to take at first, then it reverts back to the old config. This
>> is super annoying, obviously. :)
>>
>> So I can confirm that the APC PDUs work;
>>
>> ====
>> [root at an-a02n01 ~]# fence_apc_snmp -a 10.201.2.3 -n 3 -o status
>> Status: ON
>> [root at an-a02n01 ~]# fence_apc_snmp -a 10.201.2.4 -n 3 -o status
>> Status: ON
>> ====
>>
>> Here's the config as-shipped;
>>
>> ====
>> # pcs stonith config apc_snmp_node1_an-pdu03
>>  Resource: apc_snmp_node1_an-pdu03 (class=stonith type=fence_apc_snmp)
>>   Attributes: ip=10.201.2.1 pcmk_host_list=an-a02n01
>> pcmk_off_action=reboot port=5
>>   Operations: monitor interval=60
>> (apc_snmp_node1_an-pdu03-monitor-interval-60)
>>  Target: an-a02n01
>>    Level 1 - ipmilan_node1
>>    Level 2 - apc_snmp_node1_an-pdu03,apc_snmp_node1_an-pdu04
>>    Level 3 - delay_node1
>>  Target: an-a02n02
>>    Level 1 - ipmilan_node2
>>    Level 2 - apc_snmp_node2_an-pdu03,apc_snmp_node2_an-pdu04
>>    Level 3 - delay_node2
>> ====
>>
>> So in this example, I am trying to update 'apc_snmp_node1_an-pdu03' to
>> change the IP from 10.201.2.1 -> 10.201.2.3 and to change the port from
>> port 5 -> 3.
>>
>> ====
>> # pcs stonith update apc_snmp_node1_an-pdu03 ip=10.201.2.3
>> pcmk_host_list=an-a02n01 pcmk_off_action=reboot port=3
>> # pcs stonith config apc_snmp_node1_an-pdu03
>>  Resource: apc_snmp_node1_an-pdu03 (class=stonith type=fence_apc_snmp)
>>   Attributes: ip=10.201.2.3 pcmk_host_list=an-a02n01
>> pcmk_off_action=reboot port=3
>> ====
>>
>> As you can see, initially it appears to work. However, after a minute,
>> the config reverts;
>>
>> ====
>> # pcs stonith config apc_snmp_node1_an-pdu03
>>  Resource: apc_snmp_node1_an-pdu03 (class=stonith type=fence_apc_snmp)
>>   Attributes: ip=10.201.2.1 pcmk_host_list=an-a02n01
>> pcmk_off_action=reboot port=5
>> ====
>>
>> This happens to all four stonith devices (two per node, two nodes). I've
>> tried doing a 'pcs stonith disable <id>' for all four devices, and did a
>> 'pcs stonith cleanup' to make sure errors were cleared before updating,
>> still no luck.
>>
>> Any idea what I'm doing wrong?
>>
>> The logs from the node I run the update on, followed by the logs on the
>> peer:
>>
>> digimer
>>
>> ====
>> Jul 23 16:44:43 an-a02n01.alteeve.com pacemaker-attrd[121631]:  notice:
>> Updating all attributes after cib_refresh_notify event
>> Jul 23 16:44:43 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: State transition S_IDLE -> S_POLICY_ENGINE
>> Jul 23 16:44:43 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: State transition S_ELECTION -> S_INTEGRATION
>> Jul 23 16:44:43 an-a02n01.alteeve.com pacemaker-fenced[121629]:  notice:
>> Added 'apc_snmp_node1_an-pdu03' to device list (5 active devices)
>> Jul 23 16:44:47 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> notice: Clearing failure of apc_snmp_node1_an-pdu03 on an-a02n01 because
>> resource parameters have changed
>> Jul 23 16:44:47 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node1_an-pdu03 on an-a02n01 at Jul 23 16:37:55 2021
>> Jul 23 16:44:47 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu03 on an-a02n01 at Jul 23 16:35:04 2021
>> Jul 23 16:44:47 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu04 on an-a02n01 at Jul 23 16:35:04 2021
>> Jul 23 16:44:47 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node1_an-pdu04 on an-a02n02 at Jul 23 16:34:50 2021
>> Jul 23 16:44:47 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> notice: Clearing failure of apc_snmp_node1_an-pdu03 on an-a02n02 because
>> resource parameters have changed
>> Jul 23 16:44:47 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node1_an-pdu03 on an-a02n02 at Jul 23 16:37:42 2021
>> Jul 23 16:44:47 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu03 on an-a02n02 at Jul 23 16:34:50 2021
>> Jul 23 16:44:47 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu04 on an-a02n02 at Jul 23 16:34:50 2021
>> Jul 23 16:44:47 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu03 away from an-a02n01 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:44:47 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n01 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:44:47 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node1_an-pdu04 away from an-a02n02 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:44:47 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu03 away from an-a02n02 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:44:47 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n02 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:44:47 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> notice: Actions: Start      apc_snmp_node1_an-pdu03     (             
>> an-a02n02 )
>> Jul 23 16:44:47 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> notice: Calculated transition 173, saving inputs in
>> /var/lib/pacemaker/pengine/pe-input-193.bz2
>> Jul 23 16:44:47 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: Initiating start operation apc_snmp_node1_an-pdu03_start_0 on
>> an-a02n02
>> Jul 23 16:44:47 an-a02n01.alteeve.com pacemaker-attrd[121631]:  notice:
>> Setting fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n01]: INFINITY
>> -> (unset)
>> Jul 23 16:44:47 an-a02n01.alteeve.com pacemaker-attrd[121631]:  notice:
>> Setting last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n01]:
>> 1627083488 -> (unset)
>> Jul 23 16:44:47 an-a02n01.alteeve.com pacemaker-attrd[121631]:  notice:
>> Setting fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n02]: INFINITY
>> -> (unset)
>> Jul 23 16:44:47 an-a02n01.alteeve.com pacemaker-attrd[121631]:  notice:
>> Setting last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n02]:
>> 1627083475 -> (unset)
>> Jul 23 16:44:47 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: Transition 173 aborted by deletion of
>> lrm_rsc_op[@id='apc_snmp_node1_an-pdu03_last_failure_0']: Resource
>> operation removal
>> Jul 23 16:44:48 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: Transition 173 (Complete=3, Pending=0, Fired=0, Skipped=1,
>> Incomplete=1, Source=/var/lib/pacemaker/pengine/pe-input-193.bz2): Stopped
>> Jul 23 16:44:48 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu03 on an-a02n01 at Jul 23 16:35:04 2021
>> Jul 23 16:44:48 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu04 on an-a02n01 at Jul 23 16:35:04 2021
>> Jul 23 16:44:48 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node1_an-pdu04 on an-a02n02 at Jul 23 16:34:50 2021
>> Jul 23 16:44:48 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu03 on an-a02n02 at Jul 23 16:34:50 2021
>> Jul 23 16:44:48 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu04 on an-a02n02 at Jul 23 16:34:50 2021
>> Jul 23 16:44:48 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu03 away from an-a02n01 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:44:48 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n01 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:44:48 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node1_an-pdu04 away from an-a02n02 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:44:48 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu03 away from an-a02n02 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:44:48 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n02 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:44:48 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> notice: Calculated transition 174, saving inputs in
>> /var/lib/pacemaker/pengine/pe-input-194.bz2
>> Jul 23 16:44:48 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: Initiating monitor operation
>> apc_snmp_node1_an-pdu03_monitor_60000 on an-a02n02
>> Jul 23 16:44:48 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: Transition 174 (Complete=1, Pending=0, Fired=0, Skipped=0,
>> Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-194.bz2): Complete
>> Jul 23 16:44:48 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: State transition S_TRANSITION_ENGINE -> S_IDLE
>> Jul 23 16:45:09 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: State transition S_IDLE -> S_POLICY_ENGINE
>> Jul 23 16:45:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu03 on an-a02n01 at Jul 23 16:35:04 2021
>> Jul 23 16:45:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu04 on an-a02n01 at Jul 23 16:35:04 2021
>> Jul 23 16:45:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node1_an-pdu04 on an-a02n02 at Jul 23 16:34:50 2021
>> Jul 23 16:45:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu03 on an-a02n02 at Jul 23 16:34:50 2021
>> Jul 23 16:45:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu04 on an-a02n02 at Jul 23 16:34:50 2021
>> Jul 23 16:45:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu03 away from an-a02n01 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n01 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node1_an-pdu04 away from an-a02n02 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu03 away from an-a02n02 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n02 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> notice: Actions: Stop       apc_snmp_node1_an-pdu03     (             
>> an-a02n02 )  due to node availability
>> Jul 23 16:45:09 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> notice: Calculated transition 175, saving inputs in
>> /var/lib/pacemaker/pengine/pe-input-195.bz2
>> Jul 23 16:45:09 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: Initiating stop operation apc_snmp_node1_an-pdu03_stop_0 on
>> an-a02n02
>> Jul 23 16:45:09 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: Transition 175 (Complete=1, Pending=0, Fired=0, Skipped=0,
>> Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-195.bz2): Complete
>> Jul 23 16:45:09 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: State transition S_TRANSITION_ENGINE -> S_IDLE
>> Jul 23 16:45:11 an-a02n01.alteeve.com pacemaker-attrd[121631]:  notice:
>> Updating all attributes after cib_refresh_notify event
>> Jul 23 16:45:11 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: State transition S_IDLE -> S_POLICY_ENGINE
>> Jul 23 16:45:11 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: State transition S_ELECTION -> S_INTEGRATION
>> Jul 23 16:45:11 an-a02n01.alteeve.com pacemaker-fenced[121629]:  notice:
>> Added 'apc_snmp_node1_an-pdu03' to device list (5 active devices)
>> Jul 23 16:45:16 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu03 on an-a02n01 at Jul 23 16:35:04 2021
>> Jul 23 16:45:16 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu04 on an-a02n01 at Jul 23 16:35:04 2021
>> Jul 23 16:45:16 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node1_an-pdu04 on an-a02n02 at Jul 23 16:34:50 2021
>> Jul 23 16:45:16 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu03 on an-a02n02 at Jul 23 16:34:50 2021
>> Jul 23 16:45:16 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu04 on an-a02n02 at Jul 23 16:34:50 2021
>> Jul 23 16:45:16 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu03 away from an-a02n01 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:16 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n01 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:16 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node1_an-pdu04 away from an-a02n02 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:16 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu03 away from an-a02n02 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:16 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n02 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:16 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> notice: Actions: Start      apc_snmp_node1_an-pdu03     (             
>> an-a02n02 )
>> Jul 23 16:45:16 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> notice: Calculated transition 176, saving inputs in
>> /var/lib/pacemaker/pengine/pe-input-196.bz2
>> Jul 23 16:45:16 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: Initiating start operation apc_snmp_node1_an-pdu03_start_0 on
>> an-a02n02
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: Transition 176 aborted by operation
>> apc_snmp_node1_an-pdu03_start_0 'modify' on an-a02n02: Event failed
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: Transition 176 action 31 (apc_snmp_node1_an-pdu03_start_0 on
>> an-a02n02): expected 'ok' but got 'error'
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: Transition 176 (Complete=1, Pending=0, Fired=0, Skipped=0,
>> Incomplete=1, Source=/var/lib/pacemaker/pengine/pe-input-196.bz2): Complete
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-attrd[121631]:  notice:
>> Setting fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n02]: (unset)
>> -> INFINITY
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-attrd[121631]:  notice:
>> Setting last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n02]: (unset)
>> -> 1627083929
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu03 on an-a02n01 at Jul 23 16:35:04 2021
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu04 on an-a02n01 at Jul 23 16:35:04 2021
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node1_an-pdu04 on an-a02n02 at Jul 23 16:34:50 2021
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node1_an-pdu03 on an-a02n02 at Jul 23 16:45:16 2021
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node1_an-pdu03 on an-a02n02 at Jul 23 16:45:16 2021
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu03 on an-a02n02 at Jul 23 16:34:50 2021
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu04 on an-a02n02 at Jul 23 16:34:50 2021
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu03 away from an-a02n01 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n01 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node1_an-pdu04 away from an-a02n02 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu03 away from an-a02n02 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n02 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> notice: Actions: Recover    apc_snmp_node1_an-pdu03     (             
>> an-a02n02 )
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> notice: Calculated transition 177, saving inputs in
>> /var/lib/pacemaker/pengine/pe-input-197.bz2
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu03 on an-a02n01 at Jul 23 16:35:04 2021
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu04 on an-a02n01 at Jul 23 16:35:04 2021
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node1_an-pdu04 on an-a02n02 at Jul 23 16:34:50 2021
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node1_an-pdu03 on an-a02n02 at Jul 23 16:45:16 2021
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node1_an-pdu03 on an-a02n02 at Jul 23 16:45:16 2021
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu03 on an-a02n02 at Jul 23 16:34:50 2021
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu04 on an-a02n02 at Jul 23 16:34:50 2021
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu03 away from an-a02n01 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n01 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node1_an-pdu04 away from an-a02n02 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu03 away from an-a02n02 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n02 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node1_an-pdu03 away from an-a02n02 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> notice: Actions: Recover    apc_snmp_node1_an-pdu03     ( an-a02n02 ->
>> an-a02n01 )
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> notice: Calculated transition 178, saving inputs in
>> /var/lib/pacemaker/pengine/pe-input-198.bz2
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: Initiating stop operation apc_snmp_node1_an-pdu03_stop_0 on
>> an-a02n02
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: Initiating start operation apc_snmp_node1_an-pdu03_start_0
>> locally on an-a02n01
>> Jul 23 16:45:29 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: Requesting local execution of start operation for
>> apc_snmp_node1_an-pdu03 on an-a02n01
>> Jul 23 16:45:35 an-a02n01.alteeve.com pacemaker-fenced[121629]: [142B
>> blob data]
>> Jul 23 16:45:35 an-a02n01.alteeve.com pacemaker-fenced[121629]:  notice:
>> fence_apc_snmp_monitor_1[30692] error output [  ]
>> Jul 23 16:45:35 an-a02n01.alteeve.com pacemaker-fenced[121629]:  notice:
>> fence_apc_snmp_monitor_1[30692] error output [  ]
>> Jul 23 16:45:35 an-a02n01.alteeve.com pacemaker-fenced[121629]:  notice:
>> fence_apc_snmp_monitor_1[30692] error output [ 2021-07-23 16:45:35,820
>> ERROR: Please use '-h' for usage ]
>> Jul 23 16:45:35 an-a02n01.alteeve.com pacemaker-fenced[121629]:  notice:
>> fence_apc_snmp_monitor_1[30692] error output [  ]
>> Jul 23 16:45:35 an-a02n01.alteeve.com pacemaker-fenced[121629]: [128B
>> blob data]
>> Jul 23 16:45:35 an-a02n01.alteeve.com pacemaker-fenced[121629]: 
>> warning: fence_apc_snmp[30692] stderr: [  ]
>> Jul 23 16:45:35 an-a02n01.alteeve.com pacemaker-fenced[121629]: 
>> warning: fence_apc_snmp[30692] stderr: [  ]
>> Jul 23 16:45:35 an-a02n01.alteeve.com pacemaker-fenced[121629]: 
>> warning: fence_apc_snmp[30692] stderr: [ 2021-07-23 16:45:35,820 ERROR:
>> Please use '-h' for usage ]
>> Jul 23 16:45:35 an-a02n01.alteeve.com pacemaker-fenced[121629]: 
>> warning: fence_apc_snmp[30692] stderr: [  ]
>> Jul 23 16:45:42 an-a02n01.alteeve.com pacemaker-fenced[121629]: [142B
>> blob data]
>> Jul 23 16:45:42 an-a02n01.alteeve.com pacemaker-fenced[121629]:  notice:
>> fence_apc_snmp_monitor_2[31341] error output [  ]
>> Jul 23 16:45:42 an-a02n01.alteeve.com pacemaker-fenced[121629]:  notice:
>> fence_apc_snmp_monitor_2[31341] error output [  ]
>> Jul 23 16:45:42 an-a02n01.alteeve.com pacemaker-fenced[121629]:  notice:
>> fence_apc_snmp_monitor_2[31341] error output [ 2021-07-23 16:45:42,979
>> ERROR: Please use '-h' for usage ]
>> Jul 23 16:45:42 an-a02n01.alteeve.com pacemaker-fenced[121629]:  notice:
>> fence_apc_snmp_monitor_2[31341] error output [  ]
>> Jul 23 16:45:42 an-a02n01.alteeve.com pacemaker-fenced[121629]: [128B
>> blob data]
>> Jul 23 16:45:42 an-a02n01.alteeve.com pacemaker-fenced[121629]: 
>> warning: fence_apc_snmp[31341] stderr: [  ]
>> Jul 23 16:45:42 an-a02n01.alteeve.com pacemaker-fenced[121629]: 
>> warning: fence_apc_snmp[31341] stderr: [  ]
>> Jul 23 16:45:42 an-a02n01.alteeve.com pacemaker-fenced[121629]: 
>> warning: fence_apc_snmp[31341] stderr: [ 2021-07-23 16:45:42,979 ERROR:
>> Please use '-h' for usage ]
>> Jul 23 16:45:42 an-a02n01.alteeve.com pacemaker-fenced[121629]: 
>> warning: fence_apc_snmp[31341] stderr: [  ]
>> Jul 23 16:45:42 an-a02n01.alteeve.com pacemaker-fenced[121629]:  notice:
>> Operation 'monitor' [31341] using apc_snmp_node1_an-pdu03 returned -201
>> (Generic Pacemaker error)
>> Jul 23 16:45:42 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: Result of start operation for apc_snmp_node1_an-pdu03 on
>> an-a02n01: error
>> Jul 23 16:45:42 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: Transition 178 aborted by operation
>> apc_snmp_node1_an-pdu03_start_0 'modify' on an-a02n01: Event failed
>> Jul 23 16:45:42 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: Transition 178 action 32 (apc_snmp_node1_an-pdu03_start_0 on
>> an-a02n01): expected 'ok' but got 'error'
>> Jul 23 16:45:42 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: Transition 178 (Complete=2, Pending=0, Fired=0, Skipped=0,
>> Incomplete=1, Source=/var/lib/pacemaker/pengine/pe-input-198.bz2): Complete
>> Jul 23 16:45:42 an-a02n01.alteeve.com pacemaker-attrd[121631]:  notice:
>> Setting fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n01]: (unset)
>> -> INFINITY
>> Jul 23 16:45:42 an-a02n01.alteeve.com pacemaker-attrd[121631]:  notice:
>> Setting last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n01]: (unset)
>> -> 1627083942
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node1_an-pdu03 on an-a02n01 at Jul 23 16:45:29 2021
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node1_an-pdu03 on an-a02n01 at Jul 23 16:45:29 2021
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu03 on an-a02n01 at Jul 23 16:35:04 2021
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu04 on an-a02n01 at Jul 23 16:35:04 2021
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node1_an-pdu04 on an-a02n02 at Jul 23 16:34:50 2021
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node1_an-pdu03 on an-a02n02 at Jul 23 16:45:16 2021
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu03 on an-a02n02 at Jul 23 16:34:50 2021
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu04 on an-a02n02 at Jul 23 16:34:50 2021
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu03 away from an-a02n01 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n01 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node1_an-pdu04 away from an-a02n02 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu03 away from an-a02n02 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n02 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node1_an-pdu03 away from an-a02n02 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> notice: Actions: Recover    apc_snmp_node1_an-pdu03     (             
>> an-a02n01 )
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> notice: Calculated transition 179, saving inputs in
>> /var/lib/pacemaker/pengine/pe-input-199.bz2
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node1_an-pdu03 on an-a02n01 at Jul 23 16:45:29 2021
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node1_an-pdu03 on an-a02n01 at Jul 23 16:45:29 2021
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu03 on an-a02n01 at Jul 23 16:35:04 2021
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu04 on an-a02n01 at Jul 23 16:35:04 2021
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node1_an-pdu04 on an-a02n02 at Jul 23 16:34:50 2021
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node1_an-pdu03 on an-a02n02 at Jul 23 16:45:16 2021
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu03 on an-a02n02 at Jul 23 16:34:50 2021
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Unexpected result (error) was recorded for start of
>> apc_snmp_node2_an-pdu04 on an-a02n02 at Jul 23 16:34:50 2021
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu03 away from an-a02n01 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n01 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node1_an-pdu03 away from an-a02n01 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node1_an-pdu04 away from an-a02n02 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu03 away from an-a02n02 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node2_an-pdu04 away from an-a02n02 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> warning: Forcing apc_snmp_node1_an-pdu03 away from an-a02n02 after
>> 1000000 failures (max=1000000)
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> notice: Actions: Stop       apc_snmp_node1_an-pdu03     (             
>> an-a02n01 )  due to node availability
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-schedulerd[121632]: 
>> notice: Calculated transition 180, saving inputs in
>> /var/lib/pacemaker/pengine/pe-input-200.bz2
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: Initiating stop operation apc_snmp_node1_an-pdu03_stop_0 locally
>> on an-a02n01
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: Requesting local execution of stop operation for
>> apc_snmp_node1_an-pdu03 on an-a02n01
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: Result of stop operation for apc_snmp_node1_an-pdu03 on
>> an-a02n01: ok
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: Transition 180 (Complete=1, Pending=0, Fired=0, Skipped=0,
>> Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-200.bz2): Complete
>> Jul 23 16:45:43 an-a02n01.alteeve.com pacemaker-controld[121633]: 
>> notice: State transition S_TRANSITION_ENGINE -> S_IDLE
>> ====
>>
>> ====
>> Jul 23 16:44:43 an-a02n02.alteeve.com pacemaker-controld[3296860]: 
>> notice: State transition S_NOT_DC -> S_PENDING
>> Jul 23 16:44:43 an-a02n02.alteeve.com pacemaker-fenced[3296856]: 
>> notice: Added 'apc_snmp_node1_an-pdu03' to device list (5 active devices)
>> Jul 23 16:44:43 an-a02n02.alteeve.com pacemaker-controld[3296860]: 
>> notice: State transition S_PENDING -> S_NOT_DC
>> Jul 23 16:44:47 an-a02n02.alteeve.com pacemaker-controld[3296860]: 
>> notice: Requesting local execution of start operation for
>> apc_snmp_node1_an-pdu03 on an-a02n02
>> Jul 23 16:44:47 an-a02n02.alteeve.com pacemaker-attrd[3296858]:  notice:
>> Setting fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n01]: INFINITY
>> -> (unset)
>> Jul 23 16:44:47 an-a02n02.alteeve.com pacemaker-attrd[3296858]:  notice:
>> Setting last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n01]:
>> 1627083488 -> (unset)
>> Jul 23 16:44:47 an-a02n02.alteeve.com pacemaker-attrd[3296858]:  notice:
>> Setting fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n02]: INFINITY
>> -> (unset)
>> Jul 23 16:44:47 an-a02n02.alteeve.com pacemaker-attrd[3296858]:  notice:
>> Setting last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n02]:
>> 1627083475 -> (unset)
>> Jul 23 16:44:48 an-a02n02.alteeve.com pacemaker-controld[3296860]: 
>> notice: Result of start operation for apc_snmp_node1_an-pdu03 on
>> an-a02n02: ok
>> Jul 23 16:44:48 an-a02n02.alteeve.com pacemaker-controld[3296860]: 
>> notice: Requesting local execution of monitor operation for
>> apc_snmp_node1_an-pdu03 on an-a02n02
>> Jul 23 16:44:48 an-a02n02.alteeve.com pacemaker-controld[3296860]: 
>> notice: Result of monitor operation for apc_snmp_node1_an-pdu03 on
>> an-a02n02: ok
>> Jul 23 16:45:09 an-a02n02.alteeve.com pacemaker-controld[3296860]: 
>> notice: Requesting local execution of stop operation for
>> apc_snmp_node1_an-pdu03 on an-a02n02
>> Jul 23 16:45:09 an-a02n02.alteeve.com pacemaker-controld[3296860]: 
>> notice: Result of stop operation for apc_snmp_node1_an-pdu03 on
>> an-a02n02: ok
>> Jul 23 16:45:11 an-a02n02.alteeve.com pacemaker-controld[3296860]: 
>> notice: State transition S_NOT_DC -> S_PENDING
>> Jul 23 16:45:11 an-a02n02.alteeve.com pacemaker-controld[3296860]: 
>> notice: State transition S_PENDING -> S_NOT_DC
>> Jul 23 16:45:11 an-a02n02.alteeve.com pacemaker-fenced[3296856]: 
>> notice: Added 'apc_snmp_node1_an-pdu03' to device list (5 active devices)
>> Jul 23 16:45:16 an-a02n02.alteeve.com pacemaker-controld[3296860]: 
>> notice: Requesting local execution of start operation for
>> apc_snmp_node1_an-pdu03 on an-a02n02
>> Jul 23 16:45:22 an-a02n02.alteeve.com pacemaker-fenced[3296856]: [144B
>> blob data]
>> Jul 23 16:45:22 an-a02n02.alteeve.com pacemaker-fenced[3296856]: 
>> notice: fence_apc_snmp_monitor_1[3377776] error output [  ]
>> Jul 23 16:45:22 an-a02n02.alteeve.com pacemaker-fenced[3296856]: 
>> notice: fence_apc_snmp_monitor_1[3377776] error output [  ]
>> Jul 23 16:45:22 an-a02n02.alteeve.com pacemaker-fenced[3296856]: 
>> notice: fence_apc_snmp_monitor_1[3377776] error output [ 2021-07-23
>> 16:45:22,463 ERROR: Please use '-h' for usage ]
>> Jul 23 16:45:22 an-a02n02.alteeve.com pacemaker-fenced[3296856]: 
>> notice: fence_apc_snmp_monitor_1[3377776] error output [  ]
>> Jul 23 16:45:22 an-a02n02.alteeve.com pacemaker-fenced[3296856]: [130B
>> blob data]
>> Jul 23 16:45:22 an-a02n02.alteeve.com pacemaker-fenced[3296856]: 
>> warning: fence_apc_snmp[3377776] stderr: [  ]
>> Jul 23 16:45:22 an-a02n02.alteeve.com pacemaker-fenced[3296856]: 
>> warning: fence_apc_snmp[3377776] stderr: [  ]
>> Jul 23 16:45:22 an-a02n02.alteeve.com pacemaker-fenced[3296856]: 
>> warning: fence_apc_snmp[3377776] stderr: [ 2021-07-23 16:45:22,463
>> ERROR: Please use '-h' for usage ]
>> Jul 23 16:45:22 an-a02n02.alteeve.com pacemaker-fenced[3296856]: 
>> warning: fence_apc_snmp[3377776] stderr: [  ]
>> Jul 23 16:45:29 an-a02n02.alteeve.com pacemaker-fenced[3296856]: [144B
>> blob data]
>> Jul 23 16:45:29 an-a02n02.alteeve.com pacemaker-fenced[3296856]: 
>> notice: fence_apc_snmp_monitor_2[3378394] error output [  ]
>> Jul 23 16:45:29 an-a02n02.alteeve.com pacemaker-fenced[3296856]: 
>> notice: fence_apc_snmp_monitor_2[3378394] error output [  ]
>> Jul 23 16:45:29 an-a02n02.alteeve.com pacemaker-fenced[3296856]: 
>> notice: fence_apc_snmp_monitor_2[3378394] error output [ 2021-07-23
>> 16:45:29,627 ERROR: Please use '-h' for usage ]
>> Jul 23 16:45:29 an-a02n02.alteeve.com pacemaker-fenced[3296856]: 
>> notice: fence_apc_snmp_monitor_2[3378394] error output [  ]
>> Jul 23 16:45:29 an-a02n02.alteeve.com pacemaker-fenced[3296856]: [130B
>> blob data]
>> Jul 23 16:45:29 an-a02n02.alteeve.com pacemaker-fenced[3296856]: 
>> warning: fence_apc_snmp[3378394] stderr: [  ]
>> Jul 23 16:45:29 an-a02n02.alteeve.com pacemaker-fenced[3296856]: 
>> warning: fence_apc_snmp[3378394] stderr: [  ]
>> Jul 23 16:45:29 an-a02n02.alteeve.com pacemaker-fenced[3296856]: 
>> warning: fence_apc_snmp[3378394] stderr: [ 2021-07-23 16:45:29,627
>> ERROR: Please use '-h' for usage ]
>> Jul 23 16:45:29 an-a02n02.alteeve.com pacemaker-fenced[3296856]: 
>> warning: fence_apc_snmp[3378394] stderr: [  ]
>> Jul 23 16:45:29 an-a02n02.alteeve.com pacemaker-fenced[3296856]: 
>> notice: Operation 'monitor' [3378394] using apc_snmp_node1_an-pdu03
>> returned -201 (Generic Pacemaker error)
>> Jul 23 16:45:29 an-a02n02.alteeve.com pacemaker-controld[3296860]: 
>> notice: Result of start operation for apc_snmp_node1_an-pdu03 on
>> an-a02n02: error
>> Jul 23 16:45:29 an-a02n02.alteeve.com pacemaker-attrd[3296858]:  notice:
>> Setting fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n02]: (unset)
>> -> INFINITY
>> Jul 23 16:45:29 an-a02n02.alteeve.com pacemaker-attrd[3296858]:  notice:
>> Setting last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n02]: (unset)
>> -> 1627083929
>> Jul 23 16:45:29 an-a02n02.alteeve.com pacemaker-controld[3296860]: 
>> notice: Requesting local execution of stop operation for
>> apc_snmp_node1_an-pdu03 on an-a02n02
>> Jul 23 16:45:29 an-a02n02.alteeve.com pacemaker-controld[3296860]: 
>> notice: Result of stop operation for apc_snmp_node1_an-pdu03 on
>> an-a02n02: ok
>> Jul 23 16:45:42 an-a02n02.alteeve.com pacemaker-attrd[3296858]:  notice:
>> Setting fail-count-apc_snmp_node1_an-pdu03#start_0[an-a02n01]: (unset)
>> -> INFINITY
>> Jul 23 16:45:42 an-a02n02.alteeve.com pacemaker-attrd[3296858]:  notice:
>> Setting last-failure-apc_snmp_node1_an-pdu03#start_0[an-a02n01]: (unset)
>> -> 1627083942
>> ====
>>

-- 
Digimer
Papers and Projects: https://alteeve.com/w/
"I am, somehow, less interested in the weight and convolutions of Einstein’s brain than in the near certainty that people of equal talent have lived and died in cotton fields and sweatshops." - Stephen Jay Gould





More information about the Users mailing list