[ClusterLabs] [EXT] [EXT] Re: pacemaker&corosync services shutting off after the network cable is disconnected

[john tillman]

Hello again Ulrich and hello all,

Here is my configuration:

I have 3 hypervisors.  Each has 1 VM with the cluster services running 
in it.  Those are the only cluster members.
The resources are a VIP, a custom systemd service that follows the VIP, 
and mariadb service resource.
Stonith is disabled.
If there is output from a specific command that you would find helpful, 
please ask me.

The test is simply to watch the behavior of the cluster when the cable 
is removed from the hypervisor which runs the VM with the VIP resource 
started on it. When the cable is pulled, the remaining nodes behave as 
expected, meaning, the resources successfully start on other nodes.  The 
problem I saw is the unexpected exit of the pacemaker service, which you 
can see in the log file I attached.

So, my first question.  Based on the log file, can anyone tell me why 
the pacemaker service exiting?

Should pacemaker be exiting every time it loses quorum?  I ask because 
I've performed several more tests and the behavior of pacemaker service 
is inconsistent.  Sometimes, it doesn't exit.

I am very confused by this inconsistency.  Any insight would be 
appreciated.

An interesting environmental issue.  When the network cable is removed 
from the hypervisor, the load average, according to "top", start going 
up very quickly.  I mention it because it may be affecting what I see.

Thank you for your time.

Best regards,
John


On 2026-11-02 05:46, Windl, Ulrich via Users wrote:
> Hi!
> 
> I just had a short look. The problem seems to be "quorum lost". Also 
> it's not clear which of the three VMs making the cluster were affected 
> by pulling the cable from the hypervisor.
> 
> Kind regards,
> Ulrich Windl
> 
>> -----Original Message-----
>> From: Users <users-bounces at clusterlabs.org> On Behalf Of john tillman
>> Sent: Friday, October 30, 2026 1:29 PM
>> To: Cluster Labs - All topics related to open-source clustering 
>> welcomed
>> <users at clusterlabs.org>
>> Subject: [EXT] [EXT] Re: [ClusterLabs] pacemaker&corosync services 
>> shutting
>> off after the network cable is disconnected
>> 
>> Good day all!
>> 
>> So, I trimmed the file down to the point where I removed the cable 
>> from
>> the hypervisor and it is down to 111KB.  Hopefully, that is not too
>> obnoxious, and I apologize in advance for any annoyance.
>> 
>> This log is an example of what I have seen several times and 
>> represents
>> the "self exit" of pacemaker.  Seems like error status of 100 is
>> repeated.  Also, I noticed that schedulerd "is unresponsive to IPC" at
>> the top right after the resources are stopped and it never comes back.
>> 
>> I hope someone may provide some insight for what is happening or 
>> suggest
>> alternative debugging approaches.  Any help would be appreciated.
>> 
>> 3 node cluster with a VIP and a couple of services in the cluster.
>> pacemaker version 2.1.10-2
>> Rhel 9.8
>> Fencing is not enabled
>> 
>> Best regards,
>> -John
>> 
>> 
>> On 2026-10-22 15:56, john tillman wrote:
>> > Greetings Chris/All,
>> >
>> > I've had some time to get back to this issue.  It is still occurring.
>> >
>> > To refresh, I have pacemaker services running in a VM.  When I pull out
>> > the network cable from the associated hypervisor the pacemakerd process
>> > will exit.
>> >
>> > I think it would be helpful if I can provide the pacemaker.log file
>> > from when this issue occurs.  What would be the best way to provide
>> > that log file?  It's ~114KB and about 753 lines long.
>> >
>> > -John
>> >
>> >
>> > On 2026-09-14 10:07, JT wrote:
>> >> Chris,
>> >> Thanks for trying this.  My experience has been with the cluster
>> >> services running on VMs, when the cable to the hypervisor is pulled.
>> >> The hypervisor only had one VM with cluster services on it.  The
>> >> remaining cluster nodes were on other VMs on other hypervisor, all on
>> >> the same subnet.
>> >>
>> >> I'm not sure if the nmcli command being run on the VM would reproduce
>> >> the same effect?
>> >>
>> >> -John
>> >>
>> >> On September 11, 2026 4:12:43 PM EDT, Chris Lumens
>> >> <clumens at redhat.com> wrote:
>> >>
>> >>>> pacemaker 2.1.9-1    On Rhel 9.6 - No problem with network
>> >>>> isolation
>> >>>> pacemaker 2.1.10-2   On Rhel 9.8 - Problem I described
>> >>>
>> >>> I've been trying to reproduce this today, but so far no luck. I had
>> >>> an idea which patch might be to blame, so maybe there's another
>> >>> patch at fault that I haven't tried yet. Alternatively, perhaps
>> >>> there's something about how I am taking down the network on my VM
>> >>> (nmcli device down) that results in different behavior than how you
>> >>> are losing the network.
>> >>>
>> >>> - Chris
>> >> _______________________________________________
>> >> Manage your subscription:
>> >> https://lists.clusterlabs.org/mailman/listinfo/users
>> >>
>> >> ClusterLabs home: https://www.clusterlabs.org/
> 
> _______________________________________________
> Manage your subscription:
> https://lists.clusterlabs.org/mailman/listinfo/users
> 
> ClusterLabs home: https://www.clusterlabs.org/