[ClusterLabs] Antw: [EXT] How to stop removed resources when replacing cib.xml via cibadmin or crm_shadow

[Ken Gaillot]

On Thu, 2020-10-01 at 09:16 +0200, Ulrich Windl wrote:
> > > > Igor Tverdovskiy <igor.tverdovskiy.pe at gmail.com> schrieb am
> > > > 30.09.2020 um 23:49
> 
> in Nachricht
> <CAARZOoj15B37dsv+Bfx-8MMNOr4Sh6sT6SPBCeK4ex5Babau-g at mail.gmail.com>:
> > Hi All,
> > 
> > I have a necessity to apply the whole cib.xml instead of using
> > command line
> > tools like
> > > crm configure ...
> > 
> > I have generated proper XML according to predefined templates and
> > apply it
> > via
> > > cibadmin --replace --xml-file cib.xml
> 
> I think before doing such a thing you should put all nodes into
> standby, or stop all resources.
> In the past it had been a bad idea to delete running resources or
> trying to rename such.

There's no harm on the Pacemaker side in doing so.

A resource that's running but removed from the configuration is what
Pacemaker calls an "orphan". By default (the stop-orphan-resources
cluster property) it will try to stop these. Pacemaker keeps the set of
parameters that a resource was started with in memory, so it doesn't
need the now-removed configuration to perform the stop. So, the
"ORPHANED" part of this is normal and appropriate.

The problem in this particular case is the "FAILED ... (blocked)".
Removing the configuration shouldn't cause the resource to fail, and
something is blocking the stop. You should be able to see in the failed
action section of the status, or in the logs, what failed and why it's
blocked. My guess is the stop itself failed, in which case you'd need
to investigate why that happened.

To address the replacing:

Never replace the status section; it's continually updated by the live
cluster, so you could be re-uploading old information that leads to
incorrect actions. Replacing just the configuration section is the way
to go.

cibadmin --replace or crm_shadow should work fine for replacing the
configuration, but both crm and pcs allow batching commands in a file
and then applying them all at once, so it may not be necessary.

> When replacing the CIB you probably do such things.
> What are your specific reasons to replace the CIB?
> 
> > 
> > However I have encountered an issue when resources (in particular
> > VIP
> > addresses) which were completely removed/replaced by others
> > continue to run
> > on the interface after XML replacing.
> > 
> > Orphaned resources appear and VIP actually is kept started.
> > ```
> > haproxy-10.0.0.111     (ocf::haproxy):         ORPHANED FAILED
> > 738741-ip2
> > (blocked)
> > vip-10.0.0.111 (ocf::IPaddr2):         ORPHANED FAILED 738741-ip2
> > (blocked)
> > 
> > ```
> > 
> > What I have tried:
> > 1. Replace pure cib.xml without <status> element (only
> > <configuration>)
> > 2. Take active CIB from "cibadmin -Q" and replace only
> > <configuration>
> > element while keeping <status> as is.
> > 3. Create shadow replace cib.xml with/without status and commit.
> > Indeed crm_simulate -LS shows intention to stop vip-1.1.1.1, but in
> > fact it
> > will not after shadow commit.
> > 
> > Sometimes I can manage to automatically clear removed/replaced VIP
> > addresses from the interface after replacing XML, so it is
> > definitely possible, but I can not understand how to achieve this.
> > 
> > So I wonder is there a way to replace XML but at the same time stop
> > resources which are removed in a new XML?
> > 
> > Thanks!
-- 
Ken Gaillot <kgaillot at redhat.com>