Hi Reid,<div id="yMail_cursorElementTracker_1617105323709"><br></div><div id="yMail_cursorElementTracker_1617105323870">in order the colocation to work , it should (logically) look like</div><div id="yMail_cursorElementTracker_1617105348458"><br></div><div id="yMail_cursorElementTracker_1617105349117">pcs constraint colocation add BKP_IP1 with Master rsc_SAPHana_<SID>_HDB<inst.number>-clone INFINITY node-attribute=$(attribute SITE on Master <span style="font-family: sans-serif;" id="yMail_cursorElementTracker_1617105465267">rsc_SAPHana_<SID>_HDB<inst.number>-clone)</span></div><div id="yMail_cursorElementTracker_1617105349117"><font face="sans-serif"><br></font></div><div id="yMail_cursorElementTracker_1617105349117"><font face="sans-serif"><br></font></div><div id="yMail_cursorElementTracker_1617105349117"><font face="sans-serif" id="yMail_cursorElementTracker_1617105467639">So far , I have </font><span style="font-family: sans-serif;" id="yMail_cursorElementTracker_1617105500115">created a bash script to:</span></div><div id="yMail_cursorElementTracker_1617105349117"><span style="font-family: sans-serif;" id="yMail_cursorElementTracker_1617105504477">- detect global maintenance and sleep</span></div><div id="yMail_cursorElementTracker_1617105349117"><span style="font-family: sans-serif;" id="yMail_cursorElementTracker_1617105514730">- detect status without primary hana</span></div><div id="yMail_cursorElementTracker_1617105349117"><span style="" id="yMail_cursorElementTracker_1617105529991">- set node attributes (all nodes) indicating primary and secondary</span></div><div id="yMail_cursorElementTracker_1617105349117"><span style="" id="yMail_cursorElementTracker_1617105565053">- on 'cleanup' , removesall attributes</span></div><div id="yMail_cursorElementTracker_1617105349117"><span style="" id="yMail_cursorElementTracker_1617105578081"><br></span></div><div id="yMail_cursorElementTracker_1617105349117"><span style="" id="yMail_cursorElementTracker_1617105578331">That script is called via systemd service managed by pacemaker (and running with the master).</span></div><div id="yMail_cursorElementTracker_1617105349117"><span style="" id="yMail_cursorElementTracker_1617105704176"><br></span></div><div id="yMail_cursorElementTracker_1617105716263">There is a delay in the bash script and also a delay resource is defined.</div><div id="yMail_cursorElementTracker_1617105743926"><br></div><div id="yMail_cursorElementTracker_1617105745680"><br></div><div id="yMail_cursorElementTracker_1617105349117"><span style="" id="yMail_cursorElementTracker_1617105603742">All 3 IPs have a location constraints:</span></div><div id="yMail_cursorElementTracker_1617105349117"><span style="" id="yMail_cursorElementTracker_1617105634856">- Work only on specific pair</span></div><div id="yMail_cursorElementTracker_1617105349117"><span style="" id="yMail_cursorElementTracker_1617105644001">- Don't land on othernodes</span></div><div id="yMail_cursorElementTracker_1617105349117"><span style="" id="yMail_cursorElementTracker_1617105653907">- ensure -INFINITY on nodes with attribute not equal to 'primary'</span></div><div id="yMail_cursorElementTracker_1617105349117"><span style="" id="yMail_cursorElementTracker_1617105694930"><br></span></div><div id="yMail_cursorElementTracker_1617105349117"><span style="" id="yMail_cursorElementTracker_1617105695116">Order set combines the systemd service, the delay resource and all IPs (require-all=true).</span></div><div id="yMail_cursorElementTracker_1617105349117"><span style="" id="yMail_cursorElementTracker_1617105829216"><br></span></div><div id="yMail_cursorElementTracker_1617105349117"><span style="" id="yMail_cursorElementTracker_1617105829410"><br></span></div><div id="yMail_cursorElementTracker_1617105349117"><span style="" id="yMail_cursorElementTracker_1617105829618">Ugly, but it looks promising. Yet, I'm not happy to have my own scripts in the cluster's logic.</span></div><div id="yMail_cursorElementTracker_1617105349117"><span style="" id="yMail_cursorElementTracker_1617105860169"><br></span></div><div id="yMail_cursorElementTracker_1617105349117"><span style="" id="yMail_cursorElementTracker_1617105860345"><br></span></div><div id="yMail_cursorElementTracker_1617105867141">Best Regards,</div><div id="yMail_cursorElementTracker_1617105871247">Strahil Nikolov</div><div id="yMail_cursorElementTracker_1617105699445"><br></div><div id="yMail_cursorElementTracker_1617105559916"> <blockquote style="margin: 0 0 20px 0;"> <div style="color: rgb(109, 0, 246);"> <div style="">On Tue, Mar 30, 2021 at 10:06, Reid Wahl</div><div style="font-family: Roboto, sans-serif;"><nwahl@redhat.com> wrote:</div> </div> <div style="padding: 10px 0 0 20px; margin: 10px 0 0 0; border-left: 1px solid #6D00F6;"> <div id="yiv0913387909"><div><div dir="ltr"><div>You can try the following and see if it works, replacing the items in angle brackets (<>).<br clear="none"></div><div><br clear="none"></div><div> # pcs constraint colocation add <IPaddr2_rsc> with Master <SAPHanaController_rsc> INFINITY node-attribute=hana_<sid>_site</div><div><br clear="none"></div><div> However, `pcs constraint colocation add --help` gives no information about what options it accepts. It just says "[options]".<br clear="none"></div><div><br clear="none"></div><div>Usage: pcs constraint [constraints]...<br clear="none"> colocation add [<role>] <source resource id> with [<role>]<br clear="none"> <target resource id> [score] [options] [id=constraint-id]<br clear="none"> Request <source resource> to run on the same node where pacemaker has<br clear="none"> determined <target resource> should run. Positive values of score<br clear="none"> mean the resources should be run on the same node, negative values<br clear="none"> mean the resources should not be run on the same node. Specifying<br clear="none"> 'INFINITY' (or '-INFINITY') for the score forces <source resource> to<br clear="none"> run (or not run) with <target resource> (score defaults to "INFINITY").<br clear="none"> A role can be: 'Master', 'Slave', 'Started', 'Stopped' (if no role is<br clear="none"> specified, it defaults to 'Started').</div><div><br clear="none"></div><div>So it's entirely possible that pcs doesn't support creating colocation constraints with node attributes. If not, then you could edit the CIB manually and add a constraint like this:</div><div><br clear="none"></div><div> <rsc_colocation id="colocate" rsc="<IPaddr2_rsc>" with-rsc="<SAPHanaController_rsc>" score="INFINITY" with-rsc-role="Master" node-attribute="hana_<sid>_site"/></div></div><br clear="none"><div class="yiv0913387909gmail_quote"><div class="yiv0913387909gmail_attr" dir="ltr">On Mon, Mar 29, 2021 at 9:07 PM Strahil Nikolov <<a rel="nofollow noopener noreferrer" shape="rect" ymailto="mailto:hunter86_bg@yahoo.com" target="_blank" href="mailto:hunter86_bg@yahoo.com">hunter86_bg@yahoo.com</a>> wrote:<br clear="none"></div><blockquote class="yiv0913387909gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex;">Hi Ken, can you provide a prototype code example.<div id="yiv0913387909gmail-m_6866584967223492689yMail_cursorElementTracker_1617077128717"><br clear="none"></div><div id="yiv0913387909gmail-m_6866584967223492689yMail_cursorElementTracker_1617077128959">Currently,I'm making a script that will be used in a systemd service managed by the cluster.</div><div id="yiv0913387909gmail-m_6866584967223492689yMail_cursorElementTracker_1617077179372">Yet, I would like to avoid non-pacemaker solutions.</div><div id="yiv0913387909gmail-m_6866584967223492689yMail_cursorElementTracker_1617077207573"><br clear="none"></div><div id="yiv0913387909gmail-m_6866584967223492689yMail_cursorElementTracker_1617077207835">Best Regards,</div><div id="yiv0913387909gmail-m_6866584967223492689yMail_cursorElementTracker_1617077213952">Strahil Nikolov</div><div id="yiv0913387909gmail-m_6866584967223492689yMail_cursorElementTracker_1617077134044"> <br clear="none"> <blockquote style="margin:0px 0px 20px;"> <div style="font-family:Roboto, sans-serif;color:rgb(109,0,246);"> <div>On Mon, Mar 29, 2021 at 20:12, Ken Gaillot</div><div><<a rel="nofollow noopener noreferrer" shape="rect" ymailto="mailto:kgaillot@redhat.com" target="_blank" href="mailto:kgaillot@redhat.com">kgaillot@redhat.com</a>> wrote:</div> </div> <div style="padding:10px 0px 0px 20px;margin:10px 0px 0px;border-left:1px solid rgb(109,0,246);"> On Sun, 2021-03-28 at 09:20 +0300, Andrei Borzenkov wrote:<br clear="none">> On 28.03.2021 07:16, Strahil Nikolov wrote:<br clear="none">> > I didn't mean DC as a designated coordinator, but as a physical<br clear="none">> > Datecenter location.<br clear="none">> > Last time I checked, the node attributes for all nodes seemed the<br clear="none">> > same.I will verify that tomorrow (Monday).<br clear="none">> > <br clear="none">> <br clear="none">> Yes, I was probably mistaken. It is different with scale-out, agent<br clear="none">> puts<br clear="none">> information in global property section of CIB.<br clear="none">> <br clear="none">> Ideally we'd need expression that says "on node where site attribute<br clear="none">> is<br clear="none">> the same as on node where clone master is active" but I guess there<br clear="none">> is<br clear="none">> no way to express it in pacemaker.<br clear="none"><br clear="none">Yep, colocation by node attribute (combined with colocation with<br clear="none">promoted role)<br clear="none"><br clear="none"><a rel="nofollow noopener noreferrer" shape="rect" target="_blank" href="https://clusterlabs.org/pacemaker/doc/en-US/Pacemaker/2.0/html-single/Pacemaker_Explained/index.html#_colocation_properties">https://clusterlabs.org/pacemaker/doc/en-US/Pacemaker/2.0/html-single/Pacemaker_Explained/index.html#_colocation_properties</a><div id="yiv0913387909gmail-m_6866584967223492689yqtfd26131"><br clear="none"><br clear="none"><br clear="none">> <br clear="none">> I do not see any easy way to implement it without essentially<br clear="none">> duplicating SAPHanaTopology. There are some attributes that are<br clear="none">> defined<br clear="none">> but never set so far, you may try to open service request to<br clear="none">> implement<br clear="none">> consistent attribute for all nodes on current primary site.<br clear="none">> <br clear="none">> ...<br clear="none">> <br clear="none">> Hmm ... agent sets (at least, should set) hana_${SID}_vhost attribute<br clear="none">> for each node and this attribute must be unique and different between<br clear="none">> two sites. May be worth to look into it.<br clear="none">> <br clear="none">> <br clear="none">> > Best Regards,Strahil Nikolov<br clear="none">> > <br clear="none">> > <br clear="none">> > On Fri, Feb 19, 2021 at 16:51, Andrei Borzenkov<<br clear="none">> > <a rel="nofollow noopener noreferrer" shape="rect" ymailto="mailto:arvidjaar@gmail.com" target="_blank" href="mailto:arvidjaar@gmail.com">arvidjaar@gmail.com</a>> wrote: On Fri, Feb 19, 2021 at 2:44 PM<br clear="none">> > Strahil Nikolov <<a rel="nofollow noopener noreferrer" shape="rect" ymailto="mailto:hunter86_bg@yahoo.com" target="_blank" href="mailto:hunter86_bg@yahoo.com">hunter86_bg@yahoo.com</a>> wrote:<br clear="none">> > > <br clear="none">> > > <br clear="none">> > > > Do you have a fixed relation between node >pairs and VIPs? I.e.<br clear="none">> > > > must<br clear="none">> > > > A/D always get VIP1, B/E - VIP2 etc?<br clear="none">> > > <br clear="none">> > > I have to verify it again, but generally speaking - yes , VIP1 is<br clear="none">> > > always on nodeA/D (master), VIP2 on nodeB/E (worker1) , etc.<br clear="none">> > > <br clear="none">> > > I guess I can set negative constraints (-inf) -> VIP1 on node B/E<br clear="none">> > > + nodeC/F, but the stuff with the 'same DC as master' is the<br clear="none">> > > tricky part.<br clear="none">> > > <br clear="none">> > <br clear="none">> > I am not sure I understand what DC has to do with it. You have two<br clear="none">> > scale-out SAP HANA instances, one is primary, another is secondary.<br clear="none">> > If<br clear="none">> > I understand correctly your requirements, your backup application<br clear="none">> > needs to contact the primary instance which may failover to another<br clear="none">> > site. You must be using some resource agent for it, to manage<br clear="none">> > failover. The only one I am aware of is SAPHanaSR-ScaleOut. It<br clear="none">> > already<br clear="none">> > sets different node properties for primary and secondary sites.<br clear="none">> > Just<br clear="none">> > use them. If you use something else, just look at what attributes<br clear="none">> > your<br clear="none">> > RA sets. Otherwise you will be essentially duplicating your RA<br clear="none">> > functionality because you will somehow need to find out which site<br clear="none">> > is<br clear="none">> > currently primary.<br clear="none">> > <br clear="none">> > There is no guarantee that pacemaker DC wil be on the same site as<br clear="none">> > SAP<br clear="none">> > HANA primary system.</div><br clear="none">> > <br clear="none">> > <br clear="none">> <br clear="none">> _______________________________________________<br clear="none">> Manage your subscription:<br clear="none">> <a rel="nofollow noopener noreferrer" shape="rect" target="_blank" href="https://lists.clusterlabs.org/mailman/listinfo/users">https://lists.clusterlabs.org/mailman/listinfo/users</a><br clear="none">> <br clear="none">> ClusterLabs home: <a rel="nofollow noopener noreferrer" shape="rect" target="_blank" href="https://www.clusterlabs.org/">https://www.clusterlabs.org/</a><br clear="none">> <br clear="none">-- <br clear="none">Ken Gaillot <<a rel="nofollow noopener noreferrer" shape="rect" ymailto="mailto:kgaillot@redhat.com" target="_blank" href="mailto:kgaillot@redhat.com">kgaillot@redhat.com</a><div id="yiv0913387909gmail-m_6866584967223492689yqtfd76753">><br clear="none"><br clear="none"></div> </div> </blockquote></div>_______________________________________________<br clear="none">
Manage your subscription:<br clear="none">
<a rel="nofollow noopener noreferrer" shape="rect" target="_blank" href="https://lists.clusterlabs.org/mailman/listinfo/users">https://lists.clusterlabs.org/mailman/listinfo/users</a><br clear="none">
<br clear="none">
ClusterLabs home: <a rel="nofollow noopener noreferrer" shape="rect" target="_blank" href="https://www.clusterlabs.org/">https://www.clusterlabs.org/</a><div class="yiv0913387909yqt6333375156" id="yiv0913387909yqtfd75985"><br clear="none">
</div></blockquote></div><div class="yiv0913387909yqt6333375156" id="yiv0913387909yqtfd94842"><br clear="all"><br clear="none">-- <br clear="none"></div><div class="yiv0913387909gmail_signature" dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div><div class="yiv0913387909yqt6333375156" id="yiv0913387909yqtfd49755">Regards,</div><br clear="none"><br clear="none"></div>Reid Wahl, RHCA<br clear="none"></div><div>Senior Software Maintenance Engineer, Red Hat<br clear="none"></div>CEE - Platform Support Delivery - ClusterHA</div></div></div></div></div></div></div></div></div></div></div></div></div></div><div class="yiv0913387909yqt6333375156" id="yiv0913387909yqtfd85983">
</div></div></div> </div> </blockquote></div>