From chenzufei at gmail.com  Mon Mar  3 03:52:24 2025
From: chenzufei at gmail.com (chenzufei at gmail.com)
Date: Mon, 3 Mar 2025 11:52:24 +0800
Subject: [ClusterLabs] Lustre MDT/OST Mount Failures During Virtual Machine
 Reboot with Pacemaker
Message-ID: <2025030311522366665011@gmail.com>

1. Background:
There are three physical servers, each running a KVM virtual machine. The virtual machines host Lustre services (MGS/MDS/OSS). Pacemaker is used to ensure high availability of the Lustre services.
lustre(2.15.5) + corosync(3.1.5) + pacemaker(2.1.0-8.el8) + pcs(0.10.8)
2. Problem:
When a reboot command is issued on one of the virtual machines, the MDT/OST resources are taken over by the virtual machines on other nodes. However, the mounting of these resources fails during the switch (Pacemaker attempts to mount multiple times and eventually succeeds).
Workaround: Before executing the reboot command, run pcs node standby <node-name> to move the resources away.
Question: I would like to know if this is an inherent issue with Pacemaker?
3. Analysis:
From the log analysis, it appears that the MDT/OST resources are being mounted on the target node before the unmount process is completed on the source node. The Multiple Mount Protection (MMP) detects that the source node has updated the sequence number, which causes the mount operation to fail on the target node.
4. Logs:
Node 28 (Source Node):
Tue Feb 18 23:46:31 CST 2025    reboot

ll /dev/disk/by-id/virtio-ost-node28-3-36
lrwxrwxrwx 1 root root 9 Feb 18 23:47 /dev/disk/by-id/virtio-ost-node28-3-36 -> ../../vdy

Tue Feb 18 23:46:31 CST 2025
* ost-36_start_0 on lustre-oss-node29 'error' (1): call=769, status='complete', exitreason='Couldn't mount device [/dev/disk/by-id/virtio-ost-node28-3-36] as /lustre/ost-36', last-rc-change='Tue Feb 18 23:46:32 2025', queued=0ms, exec=21472ms

Feb 18 23:46:31 lustre-oss-node28 systemd[1]: Unmounting /lustre/ost-36...
Feb 18 23:46:31 lustre-oss-node28 kernel: LDISKFS-fs warning (device vdy): kmmpd:186: czf MMP failure info: epoch:6609375025013, seq: 37, last update time: 1739893591, last update node: lustre-oss-node28, last update device: vdy
Feb 18 23:46:32 lustre-oss-node28 Filesystem(ost-36)[19748]: INFO: Running stop for /dev/disk/by-id/virtio-ost-node28-3-36 on /lustre/ost-36
Feb 18 23:46:32 lustre-oss-node28 pacemaker-controld[1700]: notice: Result of stop operation for ost-36 on lustre-oss-node28: ok
Feb 18 23:46:34 lustre-oss-node28 kernel: LDISKFS-fs warning (device vdy): kmmpd:258: czf set mmp seq clean
Feb 18 23:46:34 lustre-oss-node28 kernel: LDISKFS-fs warning (device vdy): kmmpd:258: czf MMP failure info: epoch:6612033802827, seq: 4283256144, last update time: 1739893594, last update node: lustre-oss-node28, last update device: vdy
Feb 18 23:46:34 lustre-oss-node28 systemd[1]: Unmounted /lustre/ost-36.

Node 29 (Target Node):
/dev/disk/by-id/virtio-ost-node28-3-36 -> ../../vdt

Feb 18 23:46:32 lustre-oss-node29 Filesystem(ost-36)[451114]: INFO: Running start for /dev/disk/by-id/virtio-ost-node28-3-36 on /lustre/ost-36
Feb 18 23:46:32 lustre-oss-node29 kernel: LDISKFS-fs warning (device vdt): ldiskfs_multi_mount_protect:350: MMP interval 42 higher than expected, please wait.
Feb 18 23:46:53 lustre-oss-node29 kernel: czf, not equel, Current time: 23974372799987 ns, 37,4283256144
Feb 18 23:46:53 lustre-oss-node29 kernel: LDISKFS-fs warning (device vdt): ldiskfs_multi_mount_protect:364: czf MMP failure info: epoch:23974372801877, seq: 4283256144, last update time: 1739893594, last update node: lustre-oss-node28, last update device: vdy



chenzufei at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20250303/1159cadc/attachment.htm>

From oalbrigt at redhat.com  Mon Mar  3 09:03:06 2025
From: oalbrigt at redhat.com (Oyvind Albrigtsen)
Date: Mon, 3 Mar 2025 10:03:06 +0100
Subject: [ClusterLabs] Lustre MDT/OST Mount Failures During Virtual
 Machine Reboot with Pacemaker
In-Reply-To: <2025030311522366665011@gmail.com>
References: <2025030311522366665011@gmail.com>
Message-ID: <keryimnbjl6ayfshjbvhzrzlm3dyyo5enxqngrfdn3mlq7lsjx@i26hxhbxxldp>

You need the systemd drop-in functionality introduced in RHEL 9.3 to
avoid this issue:
https://bugzilla.redhat.com/show_bug.cgi?id=2184779


Oyvind

On 03/03/25 11:52 +0800, chenzufei at gmail.com wrote:
>1. Background:
>There are three physical servers, each running a KVM virtual machine. The virtual machines host Lustre services (MGS/MDS/OSS). Pacemaker is used to ensure high availability of the Lustre services.
>lustre(2.15.5) + corosync(3.1.5) + pacemaker(2.1.0-8.el8) + pcs(0.10.8)
>2. Problem:
>When a reboot command is issued on one of the virtual machines, the MDT/OST resources are taken over by the virtual machines on other nodes. However, the mounting of these resources fails during the switch (Pacemaker attempts to mount multiple times and eventually succeeds).
>Workaround: Before executing the reboot command, run pcs node standby <node-name> to move the resources away.
>Question: I would like to know if this is an inherent issue with Pacemaker?
>3. Analysis:
>From the log analysis, it appears that the MDT/OST resources are being mounted on the target node before the unmount process is completed on the source node. The Multiple Mount Protection (MMP) detects that the source node has updated the sequence number, which causes the mount operation to fail on the target node.
>4. Logs:
>Node 28 (Source Node):
>Tue Feb 18 23:46:31 CST 2025    reboot
>
>ll /dev/disk/by-id/virtio-ost-node28-3-36
>lrwxrwxrwx 1 root root 9 Feb 18 23:47 /dev/disk/by-id/virtio-ost-node28-3-36 -> ../../vdy
>
>Tue Feb 18 23:46:31 CST 2025
>* ost-36_start_0 on lustre-oss-node29 'error' (1): call=769, status='complete', exitreason='Couldn't mount device [/dev/disk/by-id/virtio-ost-node28-3-36] as /lustre/ost-36', last-rc-change='Tue Feb 18 23:46:32 2025', queued=0ms, exec=21472ms
>
>Feb 18 23:46:31 lustre-oss-node28 systemd[1]: Unmounting /lustre/ost-36...
>Feb 18 23:46:31 lustre-oss-node28 kernel: LDISKFS-fs warning (device vdy): kmmpd:186: czf MMP failure info: epoch:6609375025013, seq: 37, last update time: 1739893591, last update node: lustre-oss-node28, last update device: vdy
>Feb 18 23:46:32 lustre-oss-node28 Filesystem(ost-36)[19748]: INFO: Running stop for /dev/disk/by-id/virtio-ost-node28-3-36 on /lustre/ost-36
>Feb 18 23:46:32 lustre-oss-node28 pacemaker-controld[1700]: notice: Result of stop operation for ost-36 on lustre-oss-node28: ok
>Feb 18 23:46:34 lustre-oss-node28 kernel: LDISKFS-fs warning (device vdy): kmmpd:258: czf set mmp seq clean
>Feb 18 23:46:34 lustre-oss-node28 kernel: LDISKFS-fs warning (device vdy): kmmpd:258: czf MMP failure info: epoch:6612033802827, seq: 4283256144, last update time: 1739893594, last update node: lustre-oss-node28, last update device: vdy
>Feb 18 23:46:34 lustre-oss-node28 systemd[1]: Unmounted /lustre/ost-36.
>
>Node 29 (Target Node):
>/dev/disk/by-id/virtio-ost-node28-3-36 -> ../../vdt
>
>Feb 18 23:46:32 lustre-oss-node29 Filesystem(ost-36)[451114]: INFO: Running start for /dev/disk/by-id/virtio-ost-node28-3-36 on /lustre/ost-36
>Feb 18 23:46:32 lustre-oss-node29 kernel: LDISKFS-fs warning (device vdt): ldiskfs_multi_mount_protect:350: MMP interval 42 higher than expected, please wait.
>Feb 18 23:46:53 lustre-oss-node29 kernel: czf, not equel, Current time: 23974372799987 ns, 37,4283256144
>Feb 18 23:46:53 lustre-oss-node29 kernel: LDISKFS-fs warning (device vdt): ldiskfs_multi_mount_protect:364: czf MMP failure info: epoch:23974372801877, seq: 4283256144, last update time: 1739893594, last update node: lustre-oss-node28, last update device: vdy
>
>
>
>chenzufei at gmail.com

>_______________________________________________
>Manage your subscription:
>https://lists.clusterlabs.org/mailman/listinfo/users
>
>ClusterLabs home: https://www.clusterlabs.org/


From alma21 at gmx.at  Mon Mar  3 15:55:25 2025
From: alma21 at gmx.at (A M)
Date: Mon, 3 Mar 2025 16:55:25 +0100 (GMT+01:00)
Subject: [ClusterLabs] multiple nfs server resource groups
Message-ID: <c4b36fea-1866-4180-a161-bb645f5c1eb4@gmx.at>

Hi,

is it possible to run? 2 or more nfs (server) resource groups in parallel/HA in one cluster ??? each with it's own/unique set of nfs server (instance) , vip, VG/LV, filesystem ,....? ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20250303/85172caf/attachment.htm>

From chenzufei at gmail.com  Fri Mar 14 09:48:22 2025
From: chenzufei at gmail.com (chenzufei at gmail.com)
Date: Fri, 14 Mar 2025 17:48:22 +0800
Subject: [ClusterLabs] Investigation of Corosync Heartbeat Loss: Simulating
 Network Failures with Redundant Network Configuration
Message-ID: <2025031417480017156612@gmail.com>


Background: 
There are 11 physical machines, with two virtual machines running on each physical machine.
lustre-mds-nodexx runs the Lustre MDS server, and lustre-oss-nodexx runs the Lustre OSS service.
Each virtual machine is directly connected to two network interfaces, service1 and service2.
Pacemaker is used to ensure high availability of the Lustre services.
lustre(2.15.5) + corosync(3.1.5) + pacemaker(2.1.0-8.el8) + pcs(0.10.8)

Issue: During testing, the network interface service1 on lustre-oss-node30 and lustre-oss-node40 was repeatedly brought up and down every 1 second (to simulate a network failure).
The Corosync logs showed that heartbeats were lost, triggering a fencing action that powered off the nodes with lost heartbeats.
Given that Corosync is configured with redundant networks, why did the heartbeat loss occur? Is it due to a configuration issue, or is Corosync not designed to handle this scenario?

Other?
The configuration of corosync.conf can be found in the attached file corosync.conf.
Other relevant information is available in the attached file log.txt.
The script used for the up/down testing is attached as ip_up_and_down.sh.





chenzufei at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20250314/e36f13fe/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: log.txt
Type: application/octet-stream
Size: 25107 bytes
Desc: not available
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20250314/e36f13fe/attachment-0003.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ip_up_and_down.sh
Type: application/octet-stream
Size: 209 bytes
Desc: not available
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20250314/e36f13fe/attachment-0004.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: corosync.conf
Type: application/octet-stream
Size: 1863 bytes
Desc: not available
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20250314/e36f13fe/attachment-0005.obj>

From arvidjaar at gmail.com  Fri Mar 14 10:43:56 2025
From: arvidjaar at gmail.com (Andrei Borzenkov)
Date: Fri, 14 Mar 2025 13:43:56 +0300
Subject: [ClusterLabs] Investigation of Corosync Heartbeat Loss:
 Simulating Network Failures with Redundant Network Configuration
In-Reply-To: <2025031417480017156612@gmail.com>
References: <2025031417480017156612@gmail.com>
Message-ID: <CAA91j0V+POmQmPchQoQDsyD00B+vOrnmciTnXCxzuM1_qq8nkQ@mail.gmail.com>

On Fri, Mar 14, 2025 at 12:48?PM chenzufei at gmail.com
<chenzufei at gmail.com> wrote:
>
>
> Background:
> There are 11 physical machines, with two virtual machines running on each physical machine.
> lustre-mds-nodexx runs the Lustre MDS server, and lustre-oss-nodexx runs the Lustre OSS service.
> Each virtual machine is directly connected to two network interfaces, service1 and service2.
> Pacemaker is used to ensure high availability of the Lustre services.
> lustre(2.15.5) + corosync(3.1.5) + pacemaker(2.1.0-8.el8) + pcs(0.10.8)
>
> Issue: During testing, the network interface service1 on lustre-oss-node30 and lustre-oss-node40 was repeatedly brought up and down every 1 second (to simulate a network failure).
> The Corosync logs showed that heartbeats were lost, triggering a fencing action that powered off the nodes with lost heartbeats.
> Given that Corosync is configured with redundant networks, why did the heartbeat loss occur? Is it due to a configuration issue, or is Corosync not designed to handle this scenario?

I cannot answer this question, but the common advice on this list was
to *not* test by bringing an interface down but by blocking
communication, e.g. using netfilter (iptables/nftables).

>
> Other?
> The configuration of corosync.conf can be found in the attached file corosync.conf.
> Other relevant information is available in the attached file log.txt.
> The script used for the up/down testing is attached as ip_up_and_down.sh.
>
>
>
> ________________________________
> chenzufei at gmail.com
> _______________________________________________
> Manage your subscription:
> https://lists.clusterlabs.org/mailman/listinfo/users
>
> ClusterLabs home: https://www.clusterlabs.org/

From jfriesse at redhat.com  Fri Mar 14 11:02:30 2025
From: jfriesse at redhat.com (Jan Friesse)
Date: Fri, 14 Mar 2025 12:02:30 +0100
Subject: [ClusterLabs] Investigation of Corosync Heartbeat Loss:
 Simulating Network Failures with Redundant Network Configuration
In-Reply-To: <2025031417480017156612@gmail.com>
References: <2025031417480017156612@gmail.com>
Message-ID: <19e01d82-57d7-e46c-4383-4acbccf2230f@redhat.com>

On 14/03/2025 10:48, chenzufei at gmail.com wrote:
> 
> Background:
> There are 11 physical machines, with two virtual machines running on each physical machine.
> lustre-mds-nodexx runs the Lustre MDS server, and lustre-oss-nodexx runs the Lustre OSS service.
> Each virtual machine is directly connected to two network interfaces, service1 and service2.
> Pacemaker is used to ensure high availability of the Lustre services.
> lustre(2.15.5) + corosync(3.1.5) + pacemaker(2.1.0-8.el8) + pcs(0.10.8)
> 
> Issue: During testing, the network interface service1 on lustre-oss-node30 and lustre-oss-node40 was repeatedly brought up and down every 1 second (to simulate a network failure).
> The Corosync logs showed that heartbeats were lost, triggering a fencing action that powered off the nodes with lost heartbeats.
> Given that Corosync is configured with redundant networks, why did the heartbeat loss 

Honestly I don't think it is really configured with redundant networks.

occur? Is it due to a configuration issue, or is Corosync not designed 
to handle this scenario?

Ifdown is not ideal method of testing but Corosync 3.x should be able to 
handle it. Still using iptables/nftables/firewall is recommended.



> 
> Other?
> The configuration of corosync.conf can be found in the attached file corosync.conf.

 From config file it looks like both rings are on the same network. 
Could you please share your network configuration?

Honza

> Other relevant information is available in the attached file log.txt.
> The script used for the up/down testing is attached as ip_up_and_down.sh.
> 
> 
> 
> 
> 
> chenzufei at gmail.com
> 
> 
> _______________________________________________
> Manage your subscription:
> https://lists.clusterlabs.org/mailman/listinfo/users
> 
> ClusterLabs home: https://www.clusterlabs.org/
> 


From chenzufei at gmail.com  Sat Mar 15 03:31:30 2025
From: chenzufei at gmail.com (chenzufei at gmail.com)
Date: Sat, 15 Mar 2025 11:31:30 +0800
Subject: [ClusterLabs] Lustre MDT/OST Mount Failures During Virtual
 Machine
References: <mailman.1.1741003201.80917.users@clusterlabs.org>
Message-ID: <202503151131279039978@gmail.com>


Thank you for your advice.

The reason I understand is as follows: 
During reboot, both the system and Pacemaker will unmount the Lustre resource simultaneously. 
If the system unmounts first and Pacemaker unmounts afterward, Pacemaker will immediately return success. 
However, at this point, the system's unmounting process is not yet complete, 
causing Pacemaker to mount on the target end, which triggers this issue.

My current modification is as follows: 
Add the following lines to the file `/usr/lib/systemd/system/resource-agents-deps.target`:
```
After=remote-fs.target  
Before=shutdown.target reboot.target halt.target
```

After making this modification, the issue no longer occurs during reboot.




chenzufei at gmail.com
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20250315/6c05a103/attachment.htm>

From chenzufei at gmail.com  Sat Mar 15 03:36:08 2025
From: chenzufei at gmail.com (chenzufei at gmail.com)
Date: Sat, 15 Mar 2025 11:36:08 +0800
Subject: [ClusterLabs] Lustre MDT/OST Mount Failures During Virtual
 Machine Reboot with Pacemaker
References: <mailman.1.1741003201.80917.users@clusterlabs.org>
Message-ID: <2025031511360481435411@gmail.com>


Thank you for your advice.

The reason I understand is as follows: 
During reboot, both the system and Pacemaker will unmount the Lustre resource simultaneously. 
If the system unmounts first and Pacemaker unmounts afterward, Pacemaker will immediately return success. 
However, at this point, the system's unmounting process is not yet complete, 
causing Pacemaker to mount on the target end, which triggers this issue.

My current modification is as follows: 
Add the following lines to the file `/usr/lib/systemd/system/resource-agents-deps.target`:
```
After=remote-fs.target  
Before=shutdown.target reboot.target halt.target
```

After making this modification, the issue no longer occurs during reboot.




chenzufei at gmail.com
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20250315/d950a8dc/attachment.htm>

From piled.email at gmail.com  Sun Mar 16 12:20:04 2025
From: piled.email at gmail.com (Piled Email)
Date: Sun, 16 Mar 2025 13:20:04 +0100
Subject: [ClusterLabs] Resource placement on remote nodes with location
 constraints and transient attributes not working
Message-ID: <A5348E7F-AB84-4BF2-9D40-AD0FA2F84984@gmail.com>

I am trying to place a resource on a remote node with a location constraint that uses a transient attribute, and this does not work.

The resource is a custom resource script that starts an interface using networkctl:

primitive router-interface ocf:custom:network_interface params interface=router

The location constraint is:

location router-interface-location router-interface rule router-score: defined router-ping and router-ping gt 0

(router-ping is set by a ping resource)

The nodes:

charybdis: remote
	router-score=6000
skylla: remote
	router-score=9000

The transient attributes:

> attrd_updater -A --name router-ping
name="router-ping" host="charybdis" value="100"
name="router-ping" host="skylla" value="100"

CRM simulate output shows that the resource will not be placed anywhere.

crm_simulate -sL -Q
pcmk__primitive_assign: router-interface allocation score on argos: -INFINITY
pcmk__primitive_assign: router-interface allocation score on hestia: -INFINITY
pcmk__primitive_assign: router-interface allocation score on pylon: -INFINITY

If I use only non-transient attributes, the resource will be placed, e.g. the following places the resource:

location router-interface-location router-interface rule router-score: defined router-score

But as soon as the location constraint references transient attributes, it will not work. Pacemaker version is 2.1.8.

Am I doing something wrong here?

From u.windl at ukr.de  Mon Mar 17 09:22:28 2025
From: u.windl at ukr.de (Windl, Ulrich)
Date: Mon, 17 Mar 2025 09:22:28 +0000
Subject: [ClusterLabs] [EXT] Investigation of Corosync Heartbeat Loss:
 Simulating Network Failures with Redundant Network Configuration
In-Reply-To: <2025031417480017156612@gmail.com>
References: <2025031417480017156612@gmail.com>
Message-ID: <6cd2683c81b048af90780cd45c293ce5@ukr.de>

Looking at
    node {
        ring0_addr: 10.255.153.159
        ring1_addr: 10.255.153.160
        name: lustre-oss-node31
        nodeid: 4
    }

I wonder how the packets are routed: What is the netmask?

Kind regards,
Ulrich Windl

From: Users <users-bounces at clusterlabs.org> On Behalf Of chenzufei at gmail.com
Sent: Friday, March 14, 2025 10:48 AM
To: users <users at clusterlabs.org>
Subject: [EXT] [ClusterLabs] Investigation of Corosync Heartbeat Loss: Simulating Network Failures with Redundant Network Configuration


Background:
There are 11 physical machines, with two virtual machines running on each physical machine.
lustre-mds-nodexx runs the Lustre MDS server, and lustre-oss-nodexx runs the Lustre OSS service.
Each virtual machine is directly connected to two network interfaces, service1 and service2.
Pacemaker is used to ensure high availability of the Lustre services.
lustre(2.15.5) + corosync(3.1.5) + pacemaker(2.1.0-8.el8) + pcs(0.10.8)

Issue: During testing, the network interface service1 on lustre-oss-node30 and lustre-oss-node40 was repeatedly brought up and down every 1 second (to simulate a network failure).
The Corosync logs showed that heartbeats were lost, triggering a fencing action that powered off the nodes with lost heartbeats.
Given that Corosync is configured with redundant networks, why did the heartbeat loss occur? Is it due to a configuration issue, or is Corosync not designed to handle this scenario?

Other?
The configuration of corosync.conf can be found in the attached file corosync.conf.
Other relevant information is available in the attached file log.txt.
The script used for the up/down testing is attached as ip_up_and_down.sh.



________________________________
chenzufei at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20250317/263a3e2e/attachment.htm>

From tbean74 at gmail.com  Mon Mar 24 01:23:50 2025
From: tbean74 at gmail.com (Travis Bean)
Date: Sun, 23 Mar 2025 18:23:50 -0700
Subject: [ClusterLabs] Bash automation for Pacemaker, Corosync, DRBD, GFS,
 CLVM, and LCMC
Message-ID: <CAFk47JjD92SpFdz7XBQv+z1hyq_1pYM793o50bvVLB41A-zvuQ@mail.gmail.com>

Hello,

I developed a Bash script to automate the installation and
configuration of open-source software (i.e., launchpad.net/linuxha). I
want to make sure the syntax of this script is perfect so I can use it
as a teaching tool to educate people about Linux.

I need to know if there is anything misconfigured with my LinuxHA
high-availability Bash syntax.

Writing the Bash code to automate the installation and configuration
of Pacemaker, Corosync, DRBD, GFS, CLVM, and LCMC was a painstaking
process of trial and error to create this custom setup. I could not
find any how-to guide on the internet to show me step-by-step
instructions for what I wanted to achieve, so when I first got this
custom setup successfully working on two virtual machines, it was
really a miracle.

I haven?t been able to thoroughly test this Bash script out to make
sure it functions as intended for all environments, so I need feedback
to make sure I haven?t overlooked something with my current syntax.

If you find a bug in LinuxHA, please submit a bug report to
bugs.launchpad.net/linuxha/+filebug.

Kind regards,

Travis Bean

From Devarajan.Narayanan at dell.com  Tue Mar 25 14:37:15 2025
From: Devarajan.Narayanan at dell.com (Narayanan, Devarajan)
Date: Tue, 25 Mar 2025 14:37:15 +0000
Subject: [ClusterLabs] Queries about a Cluster setup inside a docker
Message-ID: <PH0PR19MB5178C98CD46A3899AE487C3A9FA72@PH0PR19MB5178.namprd19.prod.outlook.com>

Hi,

I have a setup where I have multiple docker instances on a base linux (Node1).
I have a cluster inside each docker instances which pair with a similar setup on another node (node2) and form 2 node clusters.
See pic1 below.

In this setup, the cluster status etc resides in the docker overlay file system I presume.
<Query1> Is there a clear list of files which have the cluster status (Basically data of corosync, pacemaker, sbd, crmsh processes I think)?

<Query2> In this setup if I wanted the cluster data to be persistent across "remove and re-run of the docker instance", what can I do?

Presuming cluster data will be in /var, /etc, /usr folders, I tried the following solution.
Created volumes for var, etc and usr and then during docker run used the options like "-v var_vol:/var -v etc_vol:/etc -v usr_vol"
With this some portion worked and saw some weird behaviour as well.
<Query3> Is this a correct way of solving the problem to have a persistent cluster data? Have I missed mapping any folder?

FYI, I have given the details about the experiment I tried to verify if the cluster data is consistent below (Experiment).
Let me know if this makes sense.


Pic1
[cid:image002.png at 01DB9DC0.635B6FA0]


Experiment
Tried the following experiment. Please let me know if this makes sense
1) In a proper working cluster, stopped app-1-on-node-2 container on node2 to get the following crm status in app-1-on-node-1
Node List:
  * Online: [ app-1-on-node-1 ]
  * OFFLINE: [ app-1-on-node-2 ]
2) Stopped and started the app-1-on-node-1 and checked the crm status. Remained the same as before
Node List:
  * Online: [ app-1-on-node-1 ]
  * OFFLINE: [ app-1-on-node-2 ]
3) Remove the container app-1-on-node-1 and run it newly and then checked the crm status.
  Now the status was changed by not showing the app-1-on-node-2 (Presume the reason for this is old cluster data is not available)
Node List:
  * Online: [ app-1-on-node-1 ]
4) Repeated the step 1 and observed the crm status (This time I used "-v var_vol:/var -v etc_vol:/etc -v usr_vol" during docker run)
Node List:
  * Online: [ app-1-on-node-1 ]
  * OFFLINE: [ app-1-on-node-2 ]
5) Remove the container app-1-on-node-1 and run it newly (with "-v var_vol:/var -v etc_vol:/etc -v usr_vol" during docker run) and then checked the crm status.

6) Now checked the crm status
Node List:
  * Online: [ app-1-on-node-1 ]
  * OFFLINE: [ app-1-on-node-2 ]

Regards,
Deva.


Internal Use - Confidential
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20250325/d0b5e5c0/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 171551 bytes
Desc: image002.png
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20250325/d0b5e5c0/attachment-0001.png>

From jgdr at dalibo.com  Tue Mar 25 15:16:05 2025
From: jgdr at dalibo.com (Jehan-Guillaume de Rorthais)
Date: Tue, 25 Mar 2025 16:16:05 +0100
Subject: [ClusterLabs] Bash automation for Pacemaker, Corosync, DRBD, GFS,
 CLVM, and LCMC
In-Reply-To: <CAFk47JjD92SpFdz7XBQv+z1hyq_1pYM793o50bvVLB41A-zvuQ@mail.gmail.com>
References: <CAFk47JjD92SpFdz7XBQv+z1hyq_1pYM793o50bvVLB41A-zvuQ@mail.gmail.com>
Message-ID: <20250325161605.01e2859e@karst>

Hi Travis,

Le Sun, 23 Mar 2025 18:23:50 -0700,
Travis Bean <tbean74 at gmail.com> a ?crit :

> I developed a Bash script to automate the installation and
> configuration of open-source software (i.e., launchpad.net/linuxha). I
> want to make sure the syntax of this script is perfect so I can use it
> as a teaching tool to educate people about Linux.
> [?]when I first got this custom setup successfully working on two virtual
> machines, it was really a miracle.
> 
> I haven?t been able to thoroughly test this Bash script out to make
> sure it functions as intended for all environments, so I need feedback
> to make sure I haven?t overlooked something with my current syntax.

I should admit this is a huge amount of work. What a patience, cheers!

However, at least in regard with the Pacemaker/Corosync parts, it seems to me
you actually wrote in bash what tools like pcs/pcsd or crmsh are already doing
quite nicely. But, based on https://code.launchpad.net/linuxha, it seems your
script is way older than your email on this list, as I can find reference of old
technologies, your "Copyfree 2016" and indeed commits from 2016. I'm not
sure when pcs/pcsd appeared?

Anyway, nowadays, I would use Ansible and pcs/pcsd to build such clusters. It's
easier, faster, safer.

If you really want to maintain such a script for some reason, in regard with
your bash syntax, maybe you could start feed your script to "shellcheck" first.
Then, it all depend on your coding style, the bash-ism you are ready to accept,
how strict you want to be, etc?

Regards,

From s.s.sathish at ericsson.com  Thu Mar 27 13:40:01 2025
From: s.s.sathish at ericsson.com (S Sathish S)
Date: Thu, 27 Mar 2025 13:40:01 +0000
Subject: [ClusterLabs] CVE-2025-30472 mitigation need to provided newer
 version to adapt in our application
Message-ID: <DU0PR07MB8641B4B7374B761B8E21446BD5A12@DU0PR07MB8641.eurprd07.prod.outlook.com>

Hi Team,

In our application we are using below corosync version where CVE-2025-30472 are impacted and same reported in our VA tool scan,  Look like subjected CVE is mitigated with below commit message . if yes please release newer version of corosync to integrate in our system.

https://github.com/corosync/corosync/blob/73ba225cc48ebb1903897c792065cb5e876613b0/exec/totemsrp.c#L4677

Thanks and Regards,
S Sathish S
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20250327/491b961f/attachment.htm>

From jfriesse at redhat.com  Thu Mar 27 16:25:21 2025
From: jfriesse at redhat.com (Jan Friesse)
Date: Thu, 27 Mar 2025 17:25:21 +0100
Subject: [ClusterLabs] CVE-2025-30472 mitigation need to provided newer
 version to adapt in our application
In-Reply-To: <DU0PR07MB8641B4B7374B761B8E21446BD5A12@DU0PR07MB8641.eurprd07.prod.outlook.com>
References: <DU0PR07MB8641B4B7374B761B8E21446BD5A12@DU0PR07MB8641.eurprd07.prod.outlook.com>
Message-ID: <8ec896fc-d00d-fca7-be8c-96046fb4dc31@redhat.com>

Hi,

On 27/03/2025 14:40, S Sathish S via Users wrote:
> Hi Team,
> 
> In our application we are using below corosync version where CVE-2025-30472 are impacted and same reported in our VA tool scan,  Look like subjected CVE is mitigated with below commit message . if yes please release newer version of corosync to integrate in our system.

As written in comments for GH issue, the problem appears only when 
corosync runs unencrypted or if private key "leaks".

Whole situation is nicely summarized by Thomas Lamprecht:
Corosync either runs encrypted or in a trusted network, anything else, 
i.e. where this is actually a problem, is just gross negligence and 
leaks the whole cluster traffic already anyway.

Honestly I don't see any reason to release new upstream version right 
now. You can always patch corosync yourself, use kronosnet ci generated 
builds (we have them for reason) or (if using distribution version) ask 
distribution package maintainer for patched package.

Honza

> 
> https://github.com/corosync/corosync/blob/73ba225cc48ebb1903897c792065cb5e876613b0/exec/totemsrp.c#L4677
> 
> Thanks and Regards,
> S Sathish S
> 
> 
> _______________________________________________
> Manage your subscription:
> https://lists.clusterlabs.org/mailman/listinfo/users
> 
> ClusterLabs home: https://www.clusterlabs.org/
> 


From s.s.sathish at ericsson.com  Fri Mar 28 06:46:56 2025
From: s.s.sathish at ericsson.com (S Sathish S)
Date: Fri, 28 Mar 2025 06:46:56 +0000
Subject: [ClusterLabs] CVE-2025-30472 mitigation need to provided newer
 version to adapt in our application
Message-ID: <DU0PR07MB86413BFEEFD49A8D77286AAFD5A02@DU0PR07MB8641.eurprd07.prod.outlook.com>

Hi Honza/Team,

Whole situation is nicely summarized by Thomas Lamprecht:
Corosync either runs encrypted or in a trusted network, anything else, i.e. where this is actually a problem, is just gross negligence and leaks the whole cluster traffic already anyway.

Likelihood of attack: As mentioned above statement , In our application, Corosync encryption is enabled by default, then encryption key is secured and it access only superuser in the system. But somehow if private key "leaks" it will high impact entire cluster traffic.

Requesting official release for below reason:

1) Any open-source project should use official releases rather than commit-based builds.Commit-based builds may lack thorough testing and could introduce regressions or incomplete features. In contrast, official releases undergo rigorous validation, including CI/CD pipelines, unit tests, and integration tests. They also incorporate security patches and verified checksums to ensure integrity. Additionally, official releases provide detailed release notes and changelogs, simplifying change tracking and version management.
2) Adapting the Corosync security patch independently while retaining the same version (e.g., 3.1.9) is not considered an official release by the community. As a result, when the VA scan tool is executed, vulnerabilities may still be detected in the updated version.
              Reference : https://www.tenable.com/cve/CVE-2025-30472

Therefore, it is recommended to adopt the official release for CVE-2025-30472 security fixes and provide a timeline for the expected new version that includes the reported CVE fixes.

Thanks and Regards,
S Sathish
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20250328/8a01fa1c/attachment-0001.htm>

From voip at mesaproyectos.com  Fri Mar 28 06:55:54 2025
From: voip at mesaproyectos.com (VoIP)
Date: Fri, 28 Mar 2025 01:55:54 -0500
Subject: [ClusterLabs] CVE-2025-30472 mitigation need to provided newer
 version to adapt in our application
In-Reply-To: <DU0PR07MB86413BFEEFD49A8D77286AAFD5A02@DU0PR07MB8641.eurprd07.prod.outlook.com>
References: <DU0PR07MB86413BFEEFD49A8D77286AAFD5A02@DU0PR07MB8641.eurprd07.prod.outlook.com>
Message-ID: <bde196ef-63b3-4863-818c-516e7c6a0ba7@mesaproyectos.com>

-1

if you are not satisfied with the management of the software, use 
another, maybe commercial. Demanding without giving is not the 
philosophy of Open Source.

Regards

El 28/03/2025 a las 1:46 a.?m., S Sathish S via Users escribi?:
>
> Hi Honza/Team,
>
> Whole situation is nicely summarized by Thomas Lamprecht:
>
> Corosync either runs encrypted or in a trusted network, anything else, 
> i.e. where this is actually a problem, is just gross negligence and 
> leaks the whole cluster traffic already anyway.
>
> Likelihood of attack: As mentioned above statement , In our 
> application, Corosync encryption is enabled by default, then 
> encryption key is secured and it access only superuser in the system. 
> But somehow if private key "leaks" *it will high impact entire cluster 
> traffic*.
>
> Requesting official release for below reason:
>
> 1) Any open-source project should use official releases rather than 
> commit-based builds.Commit-based builds may lack thorough testing and 
> could introduce regressions or incomplete features. In contrast, 
> official releases undergo rigorous validation, including CI/CD 
> pipelines, unit tests, and integration tests. They also incorporate 
> security patches and verified checksums to ensure integrity. 
> Additionally, official releases provide detailed release notes and 
> changelogs, simplifying change tracking and version management.
>
> 2) Adapting the Corosync security patch independently while retaining 
> the same version (e.g., 3.1.9) is not considered an official release 
> by the community. As a result, when the VA scan tool is executed, 
> vulnerabilities may still be detected in the updated version.
>
> ????????????? Reference : https://www.tenable.com/cve/CVE-2025-30472
>
> Therefore, it is recommended to adopt the official release for 
> CVE-2025-30472 security fixes and *provide a timeline for the expected 
> new version that includes the reported CVE fixes*.
>
> Thanks and Regards,
>
> S Sathish
>
>
> _______________________________________________
> Manage your subscription:
> https://lists.clusterlabs.org/mailman/listinfo/users
>
> ClusterLabs home:https://www.clusterlabs.org/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20250328/50240d8d/attachment.htm>

From jfriesse at redhat.com  Fri Mar 28 08:03:00 2025
From: jfriesse at redhat.com (Jan Friesse)
Date: Fri, 28 Mar 2025 09:03:00 +0100
Subject: [ClusterLabs] CVE-2025-30472 mitigation need to provided newer
 version to adapt in our application
In-Reply-To: <DU0PR07MB86413BFEEFD49A8D77286AAFD5A02@DU0PR07MB8641.eurprd07.prod.outlook.com>
References: <DU0PR07MB86413BFEEFD49A8D77286AAFD5A02@DU0PR07MB8641.eurprd07.prod.outlook.com>
Message-ID: <e6e2d022-3ea0-fc07-e9da-cc0757017f4c@redhat.com>

On 28/03/2025 07:46, S Sathish S wrote:
> Hi Honza/Team,
> 
> Whole situation is nicely summarized by Thomas Lamprecht:
> Corosync either runs encrypted or in a trusted network, anything else, i.e. where this is actually a problem, is just gross negligence and leaks the whole cluster traffic already anyway.
> 
> Likelihood of attack: As mentioned above statement , In our application, Corosync encryption is enabled by default, then encryption key is secured and it access only superuser in the system. But somehow if private key "leaks" it will high impact entire cluster traffic.

True. If private key "leaks" it will high impact whole cluster.

Please realize this is general truth. Fixing the bug you've mentioned 
doesn't help any single bit in case of "leaked" private key.


> 
> Requesting official release for below reason:
> 
> 1) Any open-source project should use official releases rather than commit-based builds.Commit-based builds may lack thorough testing and could introduce regressions or incomplete features. In contrast, official releases undergo rigorous validation, including CI/CD pipelines, unit tests, and integration tests. They also incorporate security patches and verified checksums to ensure integrity. Additionally, official releases provide detailed release notes and changelogs, simplifying change tracking and version management.

Honestly, this looks like AI generated content. One of solutions which 
I've recommended was to use Knet CI generated builds which undergoes 
same validation for every single PR/merged commit.


> 2) Adapting the Corosync security patch independently while retaining the same version (e.g., 3.1.9) is not considered an official release by the community. As a result, when the VA scan tool is executed, vulnerabilities may still be detected in the updated version.

Ok, so you are saying RHEL/Debian/... packages which has same version 
and only increase Release part of full package version are not 
considered official release? And there is really serious vulnerability 
scan tool which checks only version part? If so, please consider move to 
something more serious.


>                Reference : https://www.tenable.com/cve/CVE-2025-30472
> 
> Therefore, it is recommended to adopt the official release for CVE-2025-30472 security fixes and provide a timeline for the expected new version that includes the reported CVE fixes.

I'm expecting to cut release later this year if nothing urgent appears. 
Also because I really want to play with the mentioned bug reproduced for 
a while to check if there is more similar bugs or not, so when new 
release is cut I can say "with some level of confidence I can say 
parsing of network data should be safe".



> 
> Thanks and Regards,
> S Sathish
> 


From kwenning at redhat.com  Fri Mar 28 08:08:13 2025
From: kwenning at redhat.com (Klaus Wenninger)
Date: Fri, 28 Mar 2025 09:08:13 +0100
Subject: [ClusterLabs] Queries about a Cluster setup inside a docker
In-Reply-To: <PH0PR19MB5178C98CD46A3899AE487C3A9FA72@PH0PR19MB5178.namprd19.prod.outlook.com>
References: <PH0PR19MB5178C98CD46A3899AE487C3A9FA72@PH0PR19MB5178.namprd19.prod.outlook.com>
Message-ID: <CALrDAo0dgt5UP+S9ZMU=R+=kpnw9ptMrj9xJ9s_E=14B4S5GOQ@mail.gmail.com>

On Tue, Mar 25, 2025 at 3:37?PM Narayanan, Devarajan via Users <
users at clusterlabs.org> wrote:

> Hi,
>
>
>
> I have a setup where I have multiple docker instances on a base linux
> (Node1).
>
> I have a cluster inside each docker instances which pair with a similar
> setup on another node (node2) and form 2 node clusters.
>
> See pic1 below.
>
>
>
> In this setup, the cluster status etc resides in the docker overlay file
> system I presume.
>
> *<Query1>* Is there a clear list of files which have the cluster status
> (Basically data of corosync, pacemaker, sbd, crmsh processes I think)?
>
>
>
> *<Query2>* In this setup if I wanted the cluster data to be persistent
> across ?remove and re-run of the docker instance?, what can I do?
>
>
>
> Presuming cluster data will be in /var, /etc, /usr folders, I tried the
> following solution.
>
> Created volumes for var, etc and usr and then during docker run used the
> options like ?-v var_vol:/var -v etc_vol:/etc -v usr_vol?
>
> With this some portion worked and saw some weird behaviour as well.
>
> *<Query3>* Is this a correct way of solving the problem to have a
> persistent cluster data? Have I missed mapping any folder?
>
>
>
> FYI, I have given the details about the experiment I tried to verify if
> the cluster data is consistent below (Experiment).
>
> Let me know if this makes sense.
>

Slightly off topic regarding your questions ...
Just out of curiosity without really having investigated what docker offers
to help with I wanted to ask how you are tackling the - quite strict -
requirement
of sbd for a watchdog-device (even if it is just softdog - which isn't
available for your setup either) to guarantee reliable self-fencing?
sbd in containers would be an interesting field to investigate and thus I'm
interested in what is there and what we could do to improve the
situation. Many years back I for instance implemented a pipe-device
offering a similar interface as the kernel-watchdog-drivers provided by
each instance of the per container lxc process (iirc the lxc processes then
had an integration with watchdog-daemon while an instance of
watchdog daemon interacting with the pipe running inside each container -
everything way before sbd appeared on the horizon - just as an example).

Regards,
Klaus

>
>
>
>
> *Pic1*
>
>
>
>
>
> *Experiment*
>
> Tried the following experiment. Please let me know if this makes sense
>
> 1) In a proper working cluster, stopped app-1-on-node-2 container on node2
> to get the following crm status in app-1-on-node-1
>
> *Node List:*
>
> *  * Online: [ app-1-on-node-1 ]*
>
> *  * OFFLINE: [ app-1-on-node-2 ]*
>
> 2) Stopped and started the app-1-on-node-1 and checked the crm status.
> Remained the same as before
>
> *Node List:*
>
> *  * Online: [ app-1-on-node-1 ]*
>
> *  * OFFLINE: [ app-1-on-node-2 ]*
>
> 3) Remove the container app-1-on-node-1 and run it newly and then checked
> the crm status.
>
>   Now the status was changed by not showing the app-1-on-node-2 (Presume
> the reason for this is old cluster data is not available)
>
> *Node List:*
>
> *  * Online: [ app-1-on-node-1 ]*
>
> 4) Repeated the step 1 and observed the crm status (This time I used ?-v
> var_vol:/var -v etc_vol:/etc -v usr_vol" during docker run)
>
> *Node List:*
>
> *  * Online: [ app-1-on-node-1 ]*
>
> *  * OFFLINE: [ app-1-on-node-2 ]*
>
> 5) Remove the container app-1-on-node-1 and run it newly (with ?-v
> var_vol:/var -v etc_vol:/etc -v usr_vol" during docker run) and then
> checked the crm status.
>
>
>
> 6) Now checked the crm status
>
> *Node List:*
>
> *  * Online: [ app-1-on-node-1 ]*
>
> *  * OFFLINE: [ app-1-on-node-2 ]*
>
>
>
> Regards,
>
> Deva.
>
> Internal Use - Confidential
> _______________________________________________
> Manage your subscription:
> https://lists.clusterlabs.org/mailman/listinfo/users
>
> ClusterLabs home: https://www.clusterlabs.org/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20250328/395c4a1d/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 171551 bytes
Desc: not available
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20250328/395c4a1d/attachment-0001.png>

From Devarajan.Narayanan at dell.com  Fri Mar 28 08:29:24 2025
From: Devarajan.Narayanan at dell.com (Narayanan, Devarajan)
Date: Fri, 28 Mar 2025 08:29:24 +0000
Subject: [ClusterLabs] Queries about a Cluster setup inside a docker
In-Reply-To: <CALrDAo0dgt5UP+S9ZMU=R+=kpnw9ptMrj9xJ9s_E=14B4S5GOQ@mail.gmail.com>
References: <PH0PR19MB5178C98CD46A3899AE487C3A9FA72@PH0PR19MB5178.namprd19.prod.outlook.com>
 <CALrDAo0dgt5UP+S9ZMU=R+=kpnw9ptMrj9xJ9s_E=14B4S5GOQ@mail.gmail.com>
Message-ID: <PH0PR19MB5178E1671097BCA46D1DE2689FA02@PH0PR19MB5178.namprd19.prod.outlook.com>

Hi Klaus,

Answer below.


how you are tackling the - quite strict - requirement
of sbd for a watchdog-device (even if it is just softdog - which isn't available for your setup either) to guarantee reliable self-fencing?
<Deva>
We have modified sbd and using similar pipe based approach communicate with base linix (watchdog_tickle modified) .
If the sbd daemon has a problem the tickle stops and a daemon in the base linux would identify it and take actions like re-start the docker container.

Regards,
Deva.



Internal Use - Confidential

From: Klaus Wenninger <kwenning at redhat.com>
Sent: 28 March 2025 13:38
To: Cluster Labs - All topics related to open-source clustering welcomed <users at clusterlabs.org>
Cc: Narayanan, Devarajan <Devarajan.Narayanan at dell.com>
Subject: Re: [ClusterLabs] Queries about a Cluster setup inside a docker


[EXTERNAL EMAIL]


On Tue, Mar 25, 2025 at 3:37?PM Narayanan, Devarajan via Users <users at clusterlabs.org<mailto:users at clusterlabs.org>> wrote:
Hi,

I have a setup where I have multiple docker instances on a base linux (Node1).
I have a cluster inside each docker instances which pair with a similar setup on another node (node2) and form 2 node clusters.
See pic1 below.

In this setup, the cluster status etc resides in the docker overlay file system I presume.
<Query1> Is there a clear list of files which have the cluster status (Basically data of corosync, pacemaker, sbd, crmsh processes I think)?

<Query2> In this setup if I wanted the cluster data to be persistent across ?remove and re-run of the docker instance?, what can I do?

Presuming cluster data will be in /var, /etc, /usr folders, I tried the following solution.
Created volumes for var, etc and usr and then during docker run used the options like ?-v var_vol:/var -v etc_vol:/etc -v usr_vol?
With this some portion worked and saw some weird behaviour as well.
<Query3> Is this a correct way of solving the problem to have a persistent cluster data? Have I missed mapping any folder?

FYI, I have given the details about the experiment I tried to verify if the cluster data is consistent below (Experiment).
Let me know if this makes sense.

Slightly off topic regarding your questions ...
Just out of curiosity without really having investigated what docker offers to help with I wanted to ask how you are tackling the - quite strict - requirement
of sbd for a watchdog-device (even if it is just softdog - which isn't available for your setup either) to guarantee reliable self-fencing?
sbd in containers would be an interesting field to investigate and thus I'm interested in what is there and what we could do to improve the
situation. Many years back I for instance implemented a pipe-device offering a similar interface as the kernel-watchdog-drivers provided by
each instance of the per container lxc process (iirc the lxc processes then had an integration with watchdog-daemon while an instance of
watchdog daemon interacting with the pipe running inside each container - everything way before sbd appeared on the horizon - just as an example).

Regards,
Klaus


Pic1
[cid:image001.png at 01DB9FE8.3160FB30]


Experiment
Tried the following experiment. Please let me know if this makes sense
1) In a proper working cluster, stopped app-1-on-node-2 container on node2 to get the following crm status in app-1-on-node-1
Node List:
  * Online: [ app-1-on-node-1 ]
  * OFFLINE: [ app-1-on-node-2 ]
2) Stopped and started the app-1-on-node-1 and checked the crm status. Remained the same as before
Node List:
  * Online: [ app-1-on-node-1 ]
  * OFFLINE: [ app-1-on-node-2 ]
3) Remove the container app-1-on-node-1 and run it newly and then checked the crm status.
  Now the status was changed by not showing the app-1-on-node-2 (Presume the reason for this is old cluster data is not available)
Node List:
  * Online: [ app-1-on-node-1 ]
4) Repeated the step 1 and observed the crm status (This time I used ?-v var_vol:/var -v etc_vol:/etc -v usr_vol" during docker run)
Node List:
  * Online: [ app-1-on-node-1 ]
  * OFFLINE: [ app-1-on-node-2 ]
5) Remove the container app-1-on-node-1 and run it newly (with ?-v var_vol:/var -v etc_vol:/etc -v usr_vol" during docker run) and then checked the crm status.

6) Now checked the crm status
Node List:
  * Online: [ app-1-on-node-1 ]
  * OFFLINE: [ app-1-on-node-2 ]

Regards,
Deva.


Internal Use - Confidential
_______________________________________________
Manage your subscription:
https://lists.clusterlabs.org/mailman/listinfo/users [lists.clusterlabs.org]<https://urldefense.com/v3/__https:/lists.clusterlabs.org/mailman/listinfo/users__;!!LpKI!mUN_tEyhzd935xq2QwiY_HDqrEQ9CEZjtAzHHca5jkRALA42VbYD_wgSGuBn2uERAcrCHMTtjmz6BdP_k_kZEds$>

ClusterLabs home: https://www.clusterlabs.org/ [clusterlabs.org]<https://urldefense.com/v3/__https:/www.clusterlabs.org/__;!!LpKI!mUN_tEyhzd935xq2QwiY_HDqrEQ9CEZjtAzHHca5jkRALA42VbYD_wgSGuBn2uERAcrCHMTtjmz6BdP_rt0VjeU$>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20250328/882c7349/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 171551 bytes
Desc: image001.png
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20250328/882c7349/attachment-0001.png>