[ClusterLabs] Bug pacemaker with multiple IP

[Ken Gaillot]

On Mon, 2022-12-19 at 09:48 +0000, Thomas CAS wrote:
> Hello Clusterlabs,
>  
> I would like to report a bug on Pacemaker with the "IPaddr2"
> resource:
>  
> OS: Debian 10
> Kernel: Linux wd-websqlng01 4.19.0-18-amd64 #1 SMP Debian 4.19.208-1
> (2021-09-29) x86_64 GNU/Linux
> Pacemaker version: 2.0.1-5+deb10u2
>  
> You will find the configuration of our cluster with 2 nodes attached.
>  
> Bug :
>  
> We have several IP configured in the cluster configuration (12)
> Sometimes the cluster is unstable with the following errors in the
> pacemaker logs:
>  
> Dec 18 21:07:51 **SENSITIVEDATA** pacemaker-execd     [5079]
> (operation_finished)   notice: NGINX-VIP-
> 232_monitor_10000:28835:stderr [
> /usr/lib/ocf/resource.d/heartbeat/IPaddr2: 1:
> /usr/lib/ocf/resource.d/heartbeat/IPaddr2: ip: Text file busy ]

This doesn't sound like a bug in the agent; "Text file busy" suggests
that the system "ip" command is being modified while the command is
running. Is a software update happening when the problem occurs?

I'm not sure whether there's some other situation that could cause that
error, but simply executing the command a bunch of times simultaneously
shouldn't cause it as far as I know.

If simultaneous monitors is somehow causing the problem, you should be
able to work around it by using different intervals for different
monitors.

> Dec 18 21:07:51 **SENSITIVEDATA** pacemaker-execd     [5079]
> (operation_finished)   notice: NGINX-VIP-
> 239_monitor_10000:28877:stderr [
> /usr/lib/ocf/resource.d/heartbeat/IPaddr2: 709:
> /usr/lib/ocf/resource.d/heartbeat/IPaddr2: ip: Text file busy ]
> Dec 18 21:07:51 **SENSITIVEDATA** pacemaker-execd     [5079]
> (operation_finished)   notice: NGINX-VIP-
> 239_monitor_10000:28877:stderr [
> /usr/lib/ocf/resource.d/heartbeat/IPaddr2: 1:
> /usr/lib/ocf/resource.d/heartbeat/IPaddr2: ip: Text file busy ]
> Dec 18 21:07:51 **SENSITIVEDATA** pacemaker-execd     [5079]
> (operation_finished)   notice: NGINX-VIP-
> 234_monitor_10000:28830:stderr [
> /usr/lib/ocf/resource.d/heartbeat/IPaddr2: 1:
> /usr/lib/ocf/resource.d/heartbeat/IPaddr2: ip: Text file busy ]
> Dec 18 21:07:51 **SENSITIVEDATA** pacemaker-execd     [5079]
> (operation_finished)   notice: NGINX-VIP-
> 231_monitor_10000:28900:stderr [
> /usr/lib/ocf/resource.d/heartbeat/IPaddr2: 709:
> /usr/lib/ocf/resource.d/heartbeat/IPaddr2: ip: Text file busy ]
> Dec 18 21:07:51 **SENSITIVEDATA** pacemaker-execd     [5079]
> (operation_finished)   notice: NGINX-VIP-
> 231_monitor_10000:28900:stderr [
> /usr/lib/ocf/resource.d/heartbeat/IPaddr2: 1:
> /usr/lib/ocf/resource.d/heartbeat/IPaddr2: ip: Text file busy ]
> Dec 18 21:07:51 **SENSITIVEDATA** pacemaker-execd     [5079]
> (operation_finished)   notice: NGINX-VIP-
> 235_monitor_10000:28905:stderr [
> /usr/lib/ocf/resource.d/heartbeat/IPaddr2: 709:
> /usr/lib/ocf/resource.d/heartbeat/IPaddr2: ip: Text file busy ]
> Dec 18 21:07:51 **SENSITIVEDATA** pacemaker-execd     [5079]
> (operation_finished)   notice: NGINX-VIP-
> 235_monitor_10000:28905:stderr [
> /usr/lib/ocf/resource.d/heartbeat/IPaddr2: 1:
> /usr/lib/ocf/resource.d/heartbeat/IPaddr2: ip: Text file busy ]
> Dec 18 21:07:51 **SENSITIVEDATA** pacemaker-execd     [5079]
> (operation_finished)   notice: NGINX-VIP-
> 237_monitor_10000:28890:stderr [
> /usr/lib/ocf/resource.d/heartbeat/IPaddr2: 709:
> /usr/lib/ocf/resource.d/heartbeat/IPaddr2: ip: Text file busy ]
> Dec 18 21:07:51 **SENSITIVEDATA** pacemaker-execd     [5079]
> (operation_finished)   notice: NGINX-VIP-
> 237_monitor_10000:28890:stderr [
> /usr/lib/ocf/resource.d/heartbeat/IPaddr2: 1:
> /usr/lib/ocf/resource.d/heartbeat/IPaddr2: ip: Text file busy ]
> Dec 18 21:07:51 **SENSITIVEDATA** pacemaker-execd     [5079]
> (operation_finished)   notice: NGINX-VIP-
> 238_monitor_10000:28876:stderr [
> /usr/lib/ocf/resource.d/heartbeat/IPaddr2: 709:
> /usr/lib/ocf/resource.d/heartbeat/IPaddr2: ip: Text file busy ]
> Dec 18 21:07:51 **SENSITIVEDATA** pacemaker-execd     [5079]
> (operation_finished)   notice: NGINX-VIP-
> 238_monitor_10000:28876:stderr [
> /usr/lib/ocf/resource.d/heartbeat/IPaddr2: 1:
> /usr/lib/ocf/resource.d/heartbeat/IPaddr2: ip: Text file busy ]
> Dec 18 21:07:51 **SENSITIVEDATA** pacemaker-execd     [5079]
> (operation_finished)   notice: NGINX-VIP_monitor_10000:28880:stderr [
> /usr/lib/ocf/resource.d/heartbeat/IPaddr2: 709:
> /usr/lib/ocf/resource.d/heartbeat/IPaddr2: ip: Text file busy ]
> Dec 18 21:07:51 **SENSITIVEDATA** pacemaker-execd     [5079]
> (operation_finished)   notice: NGINX-VIP_monitor_10000:28880:stderr [
> /usr/lib/ocf/resource.d/heartbeat/IPaddr2: 1:
> /usr/lib/ocf/resource.d/heartbeat/IPaddr2: ip: Text file busy ]
>  
> The reason is that there are a lot of IPs configured and if the
> monitors take place at the same time it causes this type of error.
>  
> Best regards,
>  
>  Thomas Cas  |  Technicien du support infogérance
>  PHONE : +33 3 51 25 23 26       WEB : www.ikoula.com/en
>  IKOULA Data Center 34 rue Pont Assy - 51100 Reims - FRANCE
>  Before printing this letter, think about the impact on the
> environment! 
>  
>         	       
>  
>  
> _______________________________________________
> Manage your subscription:
> https://lists.clusterlabs.org/mailman/listinfo/users
> 
> ClusterLabs home: https://www.clusterlabs.org/
-- 
Ken Gaillot <kgaillot at redhat.com>