[ClusterLabs] Antw: Re: Set a node attribute for multiple nodes with one command
Ken Gaillot
kgaillot at redhat.com
Thu Dec 1 16:56:27 CET 2016
On 12/01/2016 06:04 AM, Kostiantyn Ponomarenko wrote:
> OK, now I see. I still have a few questions.
> 1. Is there a good reason to not remove the attribute totally if it
> is "deleted"?
We do this for two reasons:
1. You can "delete" an attribute for just one node, while leaving it on
other nodes. Attrd needs to remember the attribute as a whole, and just
delete the value for the one node. Even deleting an attribute on all
nodes is done by separate deletes for each node, so this matters even then.
2. Information about the attribute needs to continue to exist in order
to reliably process changes from different nodes. I forget the exact
details, but I remember looking into it before.
These limitations could possibly be addressed by keeping the attribute
but setting a "deleted" flag, and changing how those would be reported,
but it's not on the radar right now.
> 2. Does "attrd_updater" sets attributes to "status" configuration
> section only?
Yes (when using corosync 2).
crm_attribute modifies the CIB directly, so there is no way to batch its
changes with a delay.
> 3. Do I need to modify/set "--delay" to 0 before removing or
> changing the attribute? Because now I see that previously set delay
> works when I delete the attribute (--delete).
It depends on what you want. The point of the delay is to write changes
out to the CIB only every so often, to save disk I/O. If you're
potentially changing attribute values several times per second, this
would be a huge performance boost. The delay affects all changes,
including deletes.
If you want a specific change to be written to the CIB immediately, then
yes, you have to set the delay to 0. You can either change the delay
beforehand with attrd_updater --update-delay or change the delay and
value together with --update-both.
> 4. Does a delay set only one time work until it's unset (set to 0)?
Yes
> Thank you,
> Kostia
>
> On Wed, Nov 30, 2016 at 10:39 PM, Ken Gaillot <kgaillot at redhat.com
> <mailto:kgaillot at redhat.com>> wrote:
>
> On 11/30/2016 11:31 AM, Kostiantyn Ponomarenko wrote:
> > Hi Ken,
> >
> > I didn't look into the logs, but I experimented with it for a while.
> > Here is what I found.
> >
> > It worked for you because this attribute - "my-attr" - has not ever been
> > set before in that cluster.
> >
> > So if you set an attribute, then remove it, and then set it with
> > "--delay", like:
> >
> > # attrd_updater -N node-0 -n my-attr --update false --delay 20
> >
> > , this delay (dampening) won't work.
>
> Once set, attributes are not truly deleted -- only their values are
> cleared. And --delay has no effect with --update if the attribute
> already exists, which is what you see above.
>
> To set a delay on an already existing attribute, you have to use
> attrd_updater --update-delay or --update-both.
>
> > Moreover, when you delete this attribute the actual remove will be
> > delayed by that "--delay" which was used when the attribute was set.
> >
> >
> > Thank you,
> > Kostia
> >
> > On Tue, Nov 29, 2016 at 1:08 AM, Ken Gaillot <kgaillot at redhat.com <mailto:kgaillot at redhat.com>
> > <mailto:kgaillot at redhat.com <mailto:kgaillot at redhat.com>>> wrote:
> >
> > On 11/24/2016 05:24 AM, Kostiantyn Ponomarenko wrote:
> > > Attribute dampening doesn't work for me also.
> > > To test that I have a script:
> > >
> > > attrd_updater -N node-0 -n my-attr --update false --delay 20
> > > sleep 3
> > > attrd_updater -N node-0 -n my-attr --update true
> > > sleep 7
> > > attrd_updater -N node-1 -n my-attr --update true
> >
> > This sequence works for me -- the attributes are not written
> to the live
> > CIB until the end of the delay, when both are written at the
> same time.
> >
> > The remaining issue must be with the policy engine. You could
> look at
> > the detail log on the DC when these changes were made; you
> should see
> > info-level messages with the CIB change with both values
> together (lines
> > with "cib_perform_op: ++" and the attribute values), then
> "Transition
> > aborted" with "Transient attribute change", then a bunch of
> "pengine:"
> > lines saying what the cluster wants to do with each resource.
> >
> > There should be some information about the scores used to
> place the
> > resources.
> >
> > >
> > > All my resources have this rule in Pacemaker config:
> > >
> > > crm configure location res1-location-rule res1 \
> > > rule 0: my-attr eq true \
> > > rule -inf: my-attr ne true
> > >
> > > On a working two-node cluster I remove "my-attr" from both
> nodes.
> > > Then run my script. And all resources start on node-0.
> > > Am I doing something wrong?
> > > Or maybe my understanding of an attribute dampening is not
> correct?
> > >
> > > My Pacemaker version is 1.1.13. (heh, not the last one, but
> it is what
> > > it is ...)
> > >
> > > Thank you,
> > > Kostia
> > >
> > > On Wed, Nov 23, 2016 at 7:27 PM, Kostiantyn Ponomarenko
> > > <konstantin.ponomarenko at gmail.com
> <mailto:konstantin.ponomarenko at gmail.com>
> > <mailto:konstantin.ponomarenko at gmail.com
> <mailto:konstantin.ponomarenko at gmail.com>>
> > > <mailto:konstantin.ponomarenko at gmail.com
> <mailto:konstantin.ponomarenko at gmail.com>
> > <mailto:konstantin.ponomarenko at gmail.com
> <mailto:konstantin.ponomarenko at gmail.com>>>> wrote:
> > >
> > > Maybe I am doing something wrong, but I cannot set "status" section
> > > node attributes to a shadow cib, cluster applies them immediately.
> > > To try it out I do in a console:
> > >
> > > crm_shadow --create test
> > > crm_attribute --type nodes --node node-0 --name my-attribute
> > > --update 1 --lifetime=reboot
> > >
> > > And this attribute is set to the live cluster configuration immediately.
> > > What am I doing wrong?
> > >
> > > Thank you,
> > > Kostia
> > >
> > > On Tue, Nov 22, 2016 at 11:33 PM, Kostiantyn Ponomarenko
> > > <konstantin.ponomarenko at gmail.com
> <mailto:konstantin.ponomarenko at gmail.com>
> > <mailto:konstantin.ponomarenko at gmail.com
> <mailto:konstantin.ponomarenko at gmail.com>>
> > > <mailto:konstantin.ponomarenko at gmail.com
> <mailto:konstantin.ponomarenko at gmail.com>
> > <mailto:konstantin.ponomarenko at gmail.com
> <mailto:konstantin.ponomarenko at gmail.com>>>> wrote:
> > >
> > > Ken,
> > > Thank you for the explanation.
> > > I will try this low-level way of shadow cib creation tomorrow.
> > > PS: I will sleep much better with this excellent news/idea. =)
> > >
> > > Thank you,
> > > Kostia
> > >
> > > On Tue, Nov 22, 2016 at 10:53 PM, Ken Gaillot
> > > <kgaillot at redhat.com <mailto:kgaillot at redhat.com>
> <mailto:kgaillot at redhat.com <mailto:kgaillot at redhat.com>>
> > <mailto:kgaillot at redhat.com <mailto:kgaillot at redhat.com>
> <mailto:kgaillot at redhat.com <mailto:kgaillot at redhat.com>>>> wrote:
> > >
> > > On 11/22/2016 04:39 AM, Kostiantyn Ponomarenko
> wrote:
> > > > Using "shadow cib" in crmsh looks like a good
> idea, but it doesn't work
> > > > with node attributes set into "status" section
> of Pacemaker config.
> > > > I wonder it it is possible to make it work
> that way.
> > >
> > > Forgot to mention -- the shadow CIB is probably
> the best way
> > > to do this.
> > > I don't know if there's a way to do it in crmsh,
> but you can
> > > use it with
> > > the low-level commands crm_shadow and crm_attribute
> > > --lifetime=reboot.
> > >
> > > > Ken,
> > > >>> start dampening timer
> > > > Could you please elaborate more on this. I
> don't get how I can set this
> > > > timer.
> > > > Do I need to set this timer for each node?
> > > >
> > > >
> > > > Thank you,
> > > > Kostia
> > > >
> > > > On Mon, Nov 21, 2016 at 9:30 AM, Ulrich Windl
> > > > <Ulrich.Windl at rz.uni-regensburg.de
> <mailto:Ulrich.Windl at rz.uni-regensburg.de>
> > <mailto:Ulrich.Windl at rz.uni-regensburg.de
> <mailto:Ulrich.Windl at rz.uni-regensburg.de>>
> > > <mailto:Ulrich.Windl at rz.uni-regensburg.de
> <mailto:Ulrich.Windl at rz.uni-regensburg.de>
> > <mailto:Ulrich.Windl at rz.uni-regensburg.de
> <mailto:Ulrich.Windl at rz.uni-regensburg.de>>>
> > > > <mailto:Ulrich.Windl at rz.uni-regensburg.de
> <mailto:Ulrich.Windl at rz.uni-regensburg.de>
> > <mailto:Ulrich.Windl at rz.uni-regensburg.de
> <mailto:Ulrich.Windl at rz.uni-regensburg.de>>
> > > <mailto:Ulrich.Windl at rz.uni-regensburg.de
> <mailto:Ulrich.Windl at rz.uni-regensburg.de>
> > <mailto:Ulrich.Windl at rz.uni-regensburg.de
> <mailto:Ulrich.Windl at rz.uni-regensburg.de>>>>> wrote:
> > > >
> > > > >>> Ken Gaillot <kgaillot at redhat.com
> <mailto:kgaillot at redhat.com> <mailto:kgaillot at redhat.com
> <mailto:kgaillot at redhat.com>>
> > > <mailto:kgaillot at redhat.com
> <mailto:kgaillot at redhat.com>
> > <mailto:kgaillot at redhat.com <mailto:kgaillot at redhat.com>>>
> <mailto:kgaillot at redhat.com <mailto:kgaillot at redhat.com>
> > <mailto:kgaillot at redhat.com <mailto:kgaillot at redhat.com>>
> > > <mailto:kgaillot at redhat.com
> <mailto:kgaillot at redhat.com> <mailto:kgaillot at redhat.com
> <mailto:kgaillot at redhat.com>>>>>
> > > > schrieb am 18.11.2016 um 16:17 in Nachricht
> > > >
> <d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com
> <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com>
> > <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com
> <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com>>
> > >
> <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com
> <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com>
> > <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com
> <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com>>>
> > > >
> > >
> > <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com
> <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com>
> > <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com
> <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com>>
> > >
> > <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com
> <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com>
> > <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com
> <mailto:d6f449da-64f8-12ad-00be-e772d8e382ca at redhat.com>>>>>:
> > > > > On 11/18/2016 08:55 AM, Kostiantyn
> Ponomarenko
> > wrote:
> > > > >> Hi folks,
> > > > >>
> > > > >> Is there a way to set a node attribute
> to the
> > > "status" section for few
> > > > >> nodes at the same time?
> > > > >>
> > > > >> In my case there is a node attribute
> which allows
> > > some resources to
> > > > >> start in the cluster if it is set.
> > > > >> If I set this node attribute for say two
> > nodes in a
> > > way - one and then
> > > > >> another, than these resources are not
> distributed
> > > equally between these
> > > > >> two nodes. That because Pacemaker picks
> the first
> > > node to with this
> > > > >> attribute is set and immediately starts all
> > allowed
> > > resources on it. And
> > > > >> this is not the behavior i would like
> to get.
> > > > >>
> > > > >> Thank you,
> > > > >> Kostia
> > > > >
> > > > > Not that I know of, but it would be a
> good feature
> > > to add to
> > > > > attrd_updater and/or crm_attribute.
> > > >
> > > > With crm (shell) you don't have
> transactions for
> > node
> > > attributes,
> > > > but for the configuration. So if you add a
> location
> > > restriction
> > > > preventing any resources on your nodes,
> then enable
> > > the nodes, and
> > > > then delete the location restrictions in one
> > > transaction, you might
> > > > get what you want. It's not elegant, but
> itt ill do.
> > > >
> > > > To the crm shell maintainer: Is is
> difficult to
> > build
> > > transactions
> > > > to node status changes? The problem I see is
> > this: For
> > > configuration
> > > > you always have transactions (requiring
> > "commit), but
> > > for nodes you
> > > > traditionally have non (effects are
> immediate). So
> > > you'd need a
> > > > thing like "start transaction" which
> requires a
> > > "commit" or some
> > > > kind of abort later.
> > > >
> > > > I also don't know whether a "shadow CIB"
> would help
> > > for the original
> > > > problem.
> > > >
> > > > Ulrich
> > > >
> > > > >
> > > > > You can probably hack it with a dampening
> > value of a
> > > few seconds. If
> > > > > your rule checks for a particular value
> of the
> > > attribute, set all the
> > > > > nodes to a different value first, which
> will write
> > > that value and
> > > > start
> > > > > the dampening timer. Then set all the
> > attributes to
> > > the desired value,
> > > > > and they will get written out together
> when the
> > > timer expires.
More information about the Users
mailing list