[ClusterLabs] Antw: [EXT] Colocation per site ?

Tue Mar 30 10:21:54 EDT 2021

On Tue, 2021-03-30 at 12:05 +0000, Strahil Nikolov wrote:
> Hi Reid,
> 
> in order the colocation to work , it should (logically) look like
> 
> pcs constraint colocation add BKP_IP1 with Master
> rsc_SAPHana_<SID>_HDB<inst.number>-clone INFINITY node-
> attribute=$(attribute SITE on
> Master rsc_SAPHana_<SID>_HDB<inst.number>-clone)

The value of node-attribute here or in CIB XML is the name of the node
attribute. It basically tells Pacemaker, "put BK_IP1 on a node whose
value for this attribute is the same as on a node running a promoted
instance of this other resource".

> 
> So far , I have created a bash script to:
> - detect global maintenance and sleep
> - detect status without primary hana
> - set node attributes (all nodes) indicating primary and secondary
> - on 'cleanup' , removesall attributes
> 
> That script is called via systemd service managed by pacemaker (and
> running with the master).
> 
> There is a delay in the bash script and also a delay resource is
> defined.

I would think the delay could be avoided by having the bash script poll
until the desired state is reached -- basically, not return ok for
start until everything is completely ready.

> All 3 IPs have a location constraints:
> - Work only on specific pair
> - Don't land on othernodes
> - ensure -INFINITY on nodes with attribute not equal to 'primary'
> 
> Order set combines the systemd service, the delay resource and all
> IPs (require-all=true).
> 
> 
> Ugly, but it looks promising. Yet, I'm not happy to have my own
> scripts in the cluster's logic.
> 
> 
> Best Regards,
> Strahil Nikolov
> 
> > On Tue, Mar 30, 2021 at 10:06, Reid Wahl
> > <nwahl at redhat.com> wrote:
> > You can try the following and see if it works, replacing the items
> > in angle brackets (<>).
> > 
> >     # pcs constraint colocation add <IPaddr2_rsc> with Master
> > <SAPHanaController_rsc> INFINITY node-attribute=hana_<sid>_site
> > 
> > However, `pcs constraint colocation add --help` gives no
> > information about what options it accepts. It just says
> > "[options]".
> > 
> > Usage: pcs constraint [constraints]...
> >     colocation add [<role>] <source resource id> with [<role>]
> >                    <target resource id> [score] [options]
> > [id=constraint-id]
> >         Request <source resource> to run on the same node where
> > pacemaker has
> >         determined <target resource> should run.  Positive values
> > of score
> >         mean the resources should be run on the same node, negative
> > values
> >         mean the resources should not be run on the same node. 
> > Specifying
> >         'INFINITY' (or '-INFINITY') for the score forces <source
> > resource> to
> >         run (or not run) with <target resource> (score defaults to
> > "INFINITY").
> >         A role can be: 'Master', 'Slave', 'Started', 'Stopped' (if
> > no role is
> >         specified, it defaults to 'Started').
> > 
> > So it's entirely possible that pcs doesn't support creating
> > colocation constraints with node attributes. If not, then you could
> > edit the CIB manually and add a constraint like this:
> > 
> >     <rsc_colocation id="colocate" rsc="<IPaddr2_rsc>" with-
> > rsc="<SAPHanaController_rsc>" score="INFINITY" with-rsc-
> > role="Master" node-attribute="hana_<sid>_site"/>
> > 
> > On Mon, Mar 29, 2021 at 9:07 PM Strahil Nikolov <
> > hunter86_bg at yahoo.com> wrote:
> > > Hi Ken, can you provide a prototype code example.
> > > 
> > > Currently,I'm making a script that will be used in a systemd
> > > service managed by the cluster.
> > > Yet, I would like to avoid non-pacemaker solutions.
> > > 
> > > Best Regards,
> > > Strahil Nikolov
> > > 
> > > > On Mon, Mar 29, 2021 at 20:12, Ken Gaillot
> > > > <kgaillot at redhat.com> wrote:
> > > > On Sun, 2021-03-28 at 09:20 +0300, Andrei Borzenkov wrote:
> > > > > On 28.03.2021 07:16, Strahil Nikolov wrote:
> > > > > > I didn't mean DC as a designated coordinator, but as a
> > > > physical
> > > > > > Datecenter location.
> > > > > > Last time I checked, the node attributes for all nodes
> > > > seemed the
> > > > > > same.I will verify that tomorrow (Monday).
> > > > > > 
> > > > > 
> > > > > Yes, I was probably mistaken. It is different with scale-out, 
> > > > agent
> > > > > puts
> > > > > information in global property section of CIB.
> > > > > 
> > > > > Ideally we'd need expression that says "on node where site
> > > > attribute
> > > > > is
> > > > > the same as on node where clone master is active" but I guess
> > > > there
> > > > > is
> > > > > no way to express it in pacemaker.
> > > > 
> > > > Yep, colocation by node attribute (combined with colocation
> > > > with
> > > > promoted role)
> > > > 
> > > > https://clusterlabs.org/pacemaker/doc/en-US/Pacemaker/2.0/html-single/Pacemaker_Explained/index.html#_colocation_properties
> > > > 
> > > > 
> > > > 
> > > > > 
> > > > > I do not see any easy way to implement it without essentially
> > > > > duplicating SAPHanaTopology. There are some attributes that
> > > > are
> > > > > defined
> > > > > but never set so far, you may try to open service request to
> > > > > implement
> > > > > consistent attribute for all nodes on current primary site.
> > > > > 
> > > > > ...
> > > > > 
> > > > > Hmm ... agent sets (at least, should set) hana_${SID}_vhost
> > > > attribute
> > > > > for each node and this attribute must be unique and different
> > > > between
> > > > > two sites. May be worth to look into it.
> > > > > 
> > > > > 
> > > > > > Best Regards,Strahil Nikolov
> > > > > >  
> > > > > >  
> > > > > >  On Fri, Feb 19, 2021 at 16:51, Andrei Borzenkov<
> > > > > > arvidjaar at gmail.com> wrote:  On Fri, Feb 19, 2021 at 2:44
> > > > PM
> > > > > > Strahil Nikolov <hunter86_bg at yahoo.com> wrote:
> > > > > > > 
> > > > > > > 
> > > > > > > > Do you have a fixed relation between node >pairs and
> > > > VIPs? I.e.
> > > > > > > > must
> > > > > > > > A/D always get VIP1, B/E - VIP2 etc?
> > > > > > > 
> > > > > > > I have to verify it again, but generally speaking - yes ,
> > > > VIP1 is
> > > > > > > always on nodeA/D (master), VIP2 on nodeB/E (worker1) ,
> > > > etc.
> > > > > > > 
> > > > > > > I guess I can set negative constraints (-inf) -> VIP1 on
> > > > node B/E
> > > > > > > + nodeC/F, but the stuff with the 'same DC as master' is
> > > > the
> > > > > > > tricky part.
> > > > > > > 
> > > > > > 
> > > > > > I am not sure I understand what DC has to do with it. You
> > > > have two
> > > > > > scale-out SAP HANA instances, one is primary, another is
> > > > secondary.
> > > > > > If
> > > > > > I understand correctly your requirements, your backup
> > > > application
> > > > > > needs to contact the primary instance which may failover to
> > > > another
> > > > > > site. You must be using some resource agent for it, to
> > > > manage
> > > > > > failover. The only one I am aware of is SAPHanaSR-ScaleOut. 
> > > > It
> > > > > > already
> > > > > > sets different node properties for primary and secondary
> > > > sites.
> > > > > > Just
> > > > > > use them. If you use something else, just look at what
> > > > attributes
> > > > > > your
> > > > > > RA sets. Otherwise you will be essentially duplicating your
> > > > RA
> > > > > > functionality because you will somehow need to find out
> > > > which site
> > > > > > is
> > > > > > currently primary.
> > > > > > 
> > > > > > There is no guarantee that pacemaker DC wil be on the same
> > > > site as
> > > > > > SAP
> > > > > > HANA primary system.
> > > > 
> > > > > >  
> > > > > > 
> > > > > 
> > > > > _______________________________________________
> > > > > Manage your subscription:
> > > > > https://lists.clusterlabs.org/mailman/listinfo/users
> > > > > 
> > > > > ClusterLabs home: https://www.clusterlabs.org/
> > > > > 
> > > > -- 
> > > > Ken Gaillot <kgaillot at redhat.com
> > > > >
> > > > 
> > > 
> > > _______________________________________________
> > > Manage your subscription:
> > > https://lists.clusterlabs.org/mailman/listinfo/users
> > > 
> > > ClusterLabs home: https://www.clusterlabs.org/
> > > 
> > 
> > 
> > -- 
> > Regards,
> > 
> > 
> > Reid Wahl, RHCA
> > Senior Software Maintenance Engineer, Red Hat
> > CEE - Platform Support Delivery - ClusterHA
-- 
Ken Gaillot <kgaillot at redhat.com>