[ClusterLabs] Antw: Re: Set a node attribute for multiple nodes with one command

Kostiantyn Ponomarenko konstantin.ponomarenko at gmail.com
Thu Dec 1 17:09:16 CET 2016


Thank you for the comprehensive answer. =)

Thank you,
Kostia

On Thu, Dec 1, 2016 at 5:56 PM, Ken Gaillot <kgaillot at redhat.com> wrote:

> On 12/01/2016 06:04 AM, Kostiantyn Ponomarenko wrote:
> > OK, now I see. I still have a few questions.
> >     1. Is there a good reason to not remove the attribute totally if it
> > is "deleted"?
>
> We do this for two reasons:
>
> 1. You can "delete" an attribute for just one node, while leaving it on
> other nodes. Attrd needs to remember the attribute as a whole, and just
> delete the value for the one node. Even deleting an attribute on all
> nodes is done by separate deletes for each node, so this matters even then.
>
> 2. Information about the attribute needs to continue to exist in order
> to reliably process changes from different nodes. I forget the exact
> details, but I remember looking into it before.
>
> These limitations could possibly be addressed by keeping the attribute
> but setting a "deleted" flag, and changing how those would be reported,
> but it's not on the radar right now.
>
> >     2. Does "attrd_updater" sets attributes to "status" configuration
> > section only?
>
> Yes (when using corosync 2).
>
> crm_attribute modifies the CIB directly, so there is no way to batch its
> changes with a delay.
>
> >     3. Do I need to modify/set "--delay" to 0 before removing or
> > changing the attribute? Because now I see that previously set delay
> > works when I delete the attribute (--delete).
>
> It depends on what you want. The point of the delay is to write changes
> out to the CIB only every so often, to save disk I/O. If you're
> potentially changing attribute values several times per second, this
> would be a huge performance boost. The delay affects all changes,
> including deletes.
>
> If you want a specific change to be written to the CIB immediately, then
> yes, you have to set the delay to 0. You can either change the delay
> beforehand with attrd_updater --update-delay or change the delay and
> value together with --update-both.
>
> >     4. Does a delay set only one time work until it's unset (set to 0)?
>
> Yes
>
> > Thank you,
> > Kostia
> >
> > On Wed, Nov 30, 2016 at 10:39 PM, Ken Gaillot <kgaillot at redhat.com
> > <mailto:kgaillot at redhat.com>> wrote:
> >
> >     On 11/30/2016 11:31 AM, Kostiantyn Ponomarenko wrote:
> >     > Hi Ken,
> >     >
> >     > I didn't look into the logs, but I experimented with it for a
> while.
> >     > Here is what I found.
> >     >
> >     > It worked for you because this attribute - "my-attr" - has not
> ever been
> >     > set before in that cluster.
> >     >
> >     > So if you set an attribute, then remove it, and then set it with
> >     > "--delay", like:
> >     >
> >     >     # attrd_updater -N node-0 -n my-attr --update false --delay 20
> >     >
> >     > , this delay (dampening) won't work.
> >
> >     Once set, attributes are not truly deleted -- only their values are
> >     cleared. And --delay has no effect with --update if the attribute
> >     already exists, which is what you see above.
> >
> >     To set a delay on an already existing attribute, you have to use
> >     attrd_updater --update-delay or --update-both.
> >
> >     > Moreover, when you delete this attribute the actual remove will be
> >     > delayed by that "--delay" which was used when the attribute was
> set.
> >     >
> >     >
> >     > Thank you,
> >     > Kostia
> >     >
> >     > On Tue, Nov 29, 2016 at 1:08 AM, Ken Gaillot <kgaillot at redhat.com
> <mailto:kgaillot at redhat.com>
> >     > <mailto:kgaillot at redhat.com <mailto:kgaillot at redhat.com>>> wrote:
> >     >
> >     >     On 11/24/2016 05:24 AM, Kostiantyn Ponomarenko wrote:
> >     >     > Attribute dampening doesn't work for me also.
> >     >     > To test that I have a script:
> >     >     >
> >     >     >     attrd_updater -N node-0 -n my-attr --update false
> --delay 20
> >     >     >     sleep 3
> >     >     >     attrd_updater -N node-0 -n my-attr --update true
> >     >     >     sleep 7
> >     >     >     attrd_updater -N node-1 -n my-attr --update true
> >     >
> >     >     This sequence works for me -- the attributes are not written
> >     to the live
> >     >     CIB until the end of the delay, when both are written at the
> >     same time.
> >     >
> >     >     The remaining issue must be with the policy engine. You could
> >     look at
> >     >     the detail log on the DC when these changes were made; you
> >     should see
> >     >     info-level messages with the CIB change with both values
> >     together (lines
> >     >     with "cib_perform_op:   ++" and the attribute values), then
> >     "Transition
> >     >     aborted" with "Transient attribute change", then a bunch of
> >     "pengine:"
> >     >     lines saying what the cluster wants to do with each resource.
> >     >
> >     >     There should be some information about the scores used to
> >     place the
> >     >     resources.
> >     >
> >     >     >
> >     >     > All my resources have this rule in Pacemaker config:
> >     >     >
> >     >     >     crm configure location res1-location-rule res1 \
> >     >     >         rule 0: my-attr eq true \
> >     >     >         rule -inf: my-attr ne true
> >     >     >
> >     >     > On a working two-node cluster I remove "my-attr" from both
> >     nodes.
> >     >     > Then run my script. And all resources start on node-0.
> >     >     > Am I doing something wrong?
> >     >     > Or maybe my understanding of an attribute dampening is not
> >     correct?
> >     >     >
> >     >     > My Pacemaker version is 1.1.13. (heh, not the last one, but
> >     it is what
> >     >     > it is ...)
> >     >     >
> >     >     > Thank you,
> >     >     > Kostia
> >     >     >
> >     >     > On Wed, Nov 23, 2016 at 7:27 PM, Kostiantyn Ponomarenko
> >     >     > <konstantin.ponomarenko at gmail.com
> >     <mailto:konstantin.ponomarenko at gmail.com>
> >     >     <mailto:konstantin.ponomarenko at gmail.com
> >     <mailto:konstantin.ponomarenko at gmail.com>>
> >     >     > <mailto:konstantin.ponomarenko at gmail.com
> >     <mailto:konstantin.ponomarenko at gmail.com>
> >     >     <mailto:konstantin.ponomarenko at gmail.com
> >     <mailto:konstantin.ponomarenko at gmail.com>>>> wrote:
> >     >     >
> >     >     >     Maybe I am doing something wrong, but I cannot set
> "status" section
> >     >     >     node attributes to a shadow cib, cluster applies them
> immediately.
> >     >     >     To try it out I do in a console:
> >     >     >
> >     >     >         crm_shadow --create test
> >     >     >         crm_attribute --type nodes --node node-0 --name
> my-attribute
> >     >     >     --update 1 --lifetime=reboot
> >     >     >
> >     >     >     And this attribute is set to the live cluster
> configuration immediately.
> >     >     >     What am I doing wrong?
> >     >     >
> >     >     >     Thank you,
> >     >     >     Kostia
> >     >     >
> >     >     >     On Tue, Nov 22, 2016 at 11:33 PM, Kostiantyn Ponomarenko
> >     >     >     <konstantin.ponomarenko at gmail.com
> >     <mailto:konstantin.ponomarenko at gmail.com>
> >     >     <mailto:konstantin.ponomarenko at gmail.com
> >     <mailto:konstantin.ponomarenko at gmail.com>>
> >     >     >     <mailto:konstantin.ponomarenko at gmail.com
> >     <mailto:konstantin.ponomarenko at gmail.com>
> >     >     <mailto:konstantin.ponomarenko at gmail.com
> >     <mailto:konstantin.ponomarenko at gmail.com>>>> wrote:
> >     >     >
> >     >     >         Ken,
> >     >     >         Thank you for the explanation.
> >     >     >         I will try this low-level way of shadow cib creation
> tomorrow.
> >     >     >         PS: I will sleep much better with this excellent
> news/idea. =)
> >     >     >
> >     >     >         Thank you,
> >     >     >         Kostia
> >     >     >
> >     >     >         On Tue, Nov 22, 2016 at 10:53 PM, Ken Gaillot
> >     >     >         <kgaillot at redhat.com <mailto:kgaillot at redhat.com>
> >     <mailto:kgaillot at redhat.com <mailto:kgaillot at redhat.com>>
> >     >     <mailto:kgaillot at redhat.com <mailto:kgaillot at redhat.com>
> >     <mailto:kgaillot at redhat.com <mailto:kgaillot at redhat.com>>>> wrote:
> >     >     >
> >     >     >             On 11/22/2016 04:39 AM, Kostiantyn Ponomarenko
> >     wrote:
> >     >     >             > Using "shadow cib" in crmsh looks like a good
> >     idea, but it doesn't work
> >     >     >             > with node attributes set into "status" section
> >     of Pacemaker config.
> >     >     >             > I wonder it it is possible to make it work
> >     that way.
> >     >     >
> >     >     >             Forgot to mention -- the shadow CIB is probably
> >     the best way
> >     >     >             to do this.
> >     >     >             I don't know if there's a way to do it in crmsh,
> >     but you can
> >     >     >             use it with
> >     >     >             the low-level commands crm_shadow and
> crm_attribute
> >     >     >             --lifetime=reboot.
> >     >     >
> >     >     >             > Ken,
> >     >     >             >>> start dampening timer
> >     >     >             > Could you please elaborate more on this. I
> >     don't get how I can set this
> >     >     >             > timer.
> >     >     >             > Do I need to set this timer for each node?
> >     >     >             >
> >     >     >             >
> >     >     >             > Thank you,
> >     >     >             > Kostia
> >     >     >             >
> >     >     >             > On Mon, Nov 21, 2016 at 9:30 AM, Ulrich Windl
> >     >     >             > <Ulrich.Windl at rz.uni-regensburg.de
> >     <mailto:Ulrich.Windl at rz.uni-regensburg.de>
> >     >     <mailto:Ulrich.Windl at rz.uni-regensburg.de
> >     <mailto:Ulrich.Windl at rz.uni-regensburg.de>>
> >     >     >             <mailto:Ulrich.Windl at rz.uni-regensburg.de
> >     <mailto:Ulrich.Windl at rz.uni-regensburg.de>
> >     >     <mailto:Ulrich.Windl at rz.uni-regensburg.de
> >     <mailto:Ulrich.Windl at rz.uni-regensburg.de>>>
> >     >     >             > <mailto:Ulrich.Windl at rz.uni-regensburg.de
> >     <mailto:Ulrich.Windl at rz.uni-regensburg.de>
> >     >     <mailto:Ulrich.Windl at rz.uni-regensburg.de
> >     <mailto:Ulrich.Windl at rz.uni-regensburg.de>>
> >     >     >             <mailto:Ulrich.Windl at rz.uni-regensburg.de
> >     <mailto:Ulrich.Windl at rz.uni-regensburg.de>
> >     >     <mailto:Ulrich.Windl at rz.uni-regensburg.de
> >     <mailto:Ulrich.Windl at rz.uni-regensburg.de>>>>> wrote:
> >     >     >             >
> >     >     >             >     >>> Ken Gaillot <kgaillot at redhat.com
> >     <mailto:kgaillot at redhat.com> <mailto:kgaillot at redhat.com
> >     <mailto:kgaillot at redhat.com>>
> >     >     >             <mailto:kgaillot at redhat.com
> >     <mailto:kgaillot at redhat.com>
> >     >     <mailto:kgaillot at redhat.com <mailto:kgaillot at redhat.com>>>
> >     <mailto:kgaillot at redhat.com <mailto:kgaillot at redhat.com>
> >     >     <mailto:kgaillot at redhat.com <mailto:kgaillot at redhat.com>>
> >     >     >             <mailto:kgaillot at redhat.com
> >     <mailto:kgaillot at redhat.com> <mailto:kgaillot at redhat.com
> >     <mailto:kgaillot at redhat.com>>>>>
> >     >     >             >     schrieb am 18.11.2016 um 16:17 in Nachricht
> >     >     >             >
> >      <d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com
> >     <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com>
> >     >     <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com
> >     <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com>>
> >     >     >
> >      <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com
> >     <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com>
> >     >     <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com
> >     <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com>>>
> >     >     >             >
> >     >     >
> >     >     <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com
> >     <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com>
> >     >     <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com
> >     <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com>>
> >     >     >
> >     >      <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com
> >     <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com>
> >     >     <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com
> >     <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com>>>>>:
> >     >     >             >     > On 11/18/2016 08:55 AM, Kostiantyn
> >     Ponomarenko
> >     >     wrote:
> >     >     >             >     >> Hi folks,
> >     >     >             >     >>
> >     >     >             >     >> Is there a way to set a node attribute
> >     to the
> >     >     >             "status" section for few
> >     >     >             >     >> nodes at the same time?
> >     >     >             >     >>
> >     >     >             >     >> In my case there is a node attribute
> >     which allows
> >     >     >             some resources to
> >     >     >             >     >> start in the cluster if it is set.
> >     >     >             >     >> If I set this node attribute for say two
> >     >     nodes in a
> >     >     >             way - one and then
> >     >     >             >     >> another, than these resources are not
> >     distributed
> >     >     >             equally between these
> >     >     >             >     >> two nodes. That because Pacemaker picks
> >     the first
> >     >     >             node to with this
> >     >     >             >     >> attribute is set and immediately starts
> all
> >     >     allowed
> >     >     >             resources on it. And
> >     >     >             >     >> this is not the behavior i would like
> >     to get.
> >     >     >             >     >>
> >     >     >             >     >> Thank you,
> >     >     >             >     >> Kostia
> >     >     >             >     >
> >     >     >             >     > Not that I know of, but it would be a
> >     good feature
> >     >     >             to add to
> >     >     >             >     > attrd_updater and/or crm_attribute.
> >     >     >             >
> >     >     >             >     With crm (shell) you don't have
> >     transactions for
> >     >     node
> >     >     >             attributes,
> >     >     >             >     but for the configuration. So if you add a
> >     location
> >     >     >             restriction
> >     >     >             >     preventing any resources on your nodes,
> >     then enable
> >     >     >             the nodes, and
> >     >     >             >     then delete the location restrictions in
> one
> >     >     >             transaction, you might
> >     >     >             >     get what you want. It's not elegant, but
> >     itt ill do.
> >     >     >             >
> >     >     >             >     To the crm shell maintainer: Is is
> >     difficult to
> >     >     build
> >     >     >             transactions
> >     >     >             >     to node status changes? The problem I see
> is
> >     >     this: For
> >     >     >             configuration
> >     >     >             >     you always have transactions (requiring
> >     >     "commit), but
> >     >     >             for nodes you
> >     >     >             >     traditionally have non (effects are
> >     immediate). So
> >     >     >             you'd need a
> >     >     >             >     thing like "start transaction" which
> >     requires a
> >     >     >             "commit" or some
> >     >     >             >     kind of abort later.
> >     >     >             >
> >     >     >             >     I also don't know whether a "shadow CIB"
> >     would help
> >     >     >             for the original
> >     >     >             >     problem.
> >     >     >             >
> >     >     >             >     Ulrich
> >     >     >             >
> >     >     >             >     >
> >     >     >             >     > You can probably hack it with a dampening
> >     >     value of a
> >     >     >             few seconds. If
> >     >     >             >     > your rule checks for a particular value
> >     of the
> >     >     >             attribute, set all the
> >     >     >             >     > nodes to a different value first, which
> >     will write
> >     >     >             that value and
> >     >     >             >     start
> >     >     >             >     > the dampening timer. Then set all the
> >     >     attributes to
> >     >     >             the desired value,
> >     >     >             >     > and they will get written out together
> >     when the
> >     >     >             timer expires.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://clusterlabs.org/pipermail/users/attachments/20161201/9a8700c6/attachment-0001.html>


More information about the Users mailing list