[ClusterLabs] pacemaker-remoted /dev/shm errors

[Christine caulfield]

Hi,

The error is coming from libqb - which is what manages the local IPC 
connections between local clients and the server.

I'm the libqb maintainer but I've never seen that error before! Is there 
anything unusual about the setup on this node? Like filesystems on NFS 
or some other networked filesystem?

Other basic things to check are that /dev/shm is not full. Yes, normally 
you'd get ENOSPC in that case but it's always worth checking because odd 
things can happen when filesystems get full.

It might be helpful strace the client and server processes when the 
error occurs (if that's possible). I'm not 100% sure which operation is 
failing with EREMOTEIO - though I can't find many useful references to 
that error in the kernel which is also slightly weird.

Chrissie

On 06/03/2023 13:03, Alexander Epaneshnikov via Users wrote:
> Hello. we are using pacemaker 2.1.4-5.el8  and seeing strange errors in the
> logs when a request is made to the cluster.
> 
> Feb 17 08:18:15 gm-srv-oshv-001.int.cld pacemaker-remoted   [2984] (handle_new_connection)      error: Error in connection setup (/dev/shm/qb-2984-1077673-18-7xR8Y0/qb): Remote I/O error (121)
> Feb 17 08:19:15 gm-srv-oshv-001.int.cld pacemaker-remoted   [2984] (handle_new_connection)      error: Error in connection setup (/dev/shm/qb-2984-1077927-18-dX5NSt/qb): Remote I/O error (121)
> Feb 17 08:20:16 gm-srv-oshv-001.int.cld pacemaker-remoted   [2984] (handle_new_connection)      error: Error in connection setup (/dev/shm/qb-2984-1078160-18-RjzD4K/qb): Remote I/O error (121)
> Feb 17 08:21:16 gm-srv-oshv-001.int.cld pacemaker-remoted   [2984] (handle_new_connection)      error: Error in connection setup (/dev/shm/qb-2984-1078400-18-YyJmJJ/qb): Remote I/O error (121)
> 
> other than that pacemaker/corosync works fine.
> 
> any suggestions on the cause of the error, or at least where to start debugging, are welcome.
>