[Pacemaker] Multi-site support in pacemaker (tokens, deadman, CTR)

[Lars Marowsky-Bree]

On 2011-04-29T03:33:00, "Gao,Yan" <ygao at novell.com> wrote:

> > Yes; a ticket section, just like that.
> All right. How about the schema:
>     <element name="configuration">
>       <interleave>
> ...
>         <element name="tickets">
>           <zeroOrMore>
>             <element name="ticket_set">
>               <externalRef href="nvset.rng"/>
>             </element>
>           </zeroOrMore>
>         </element>
> ...

Makes sense to me.

> > Personally, I lean towards this. (Andrew has expressed a wish to do
> > without the "rsc_" prefix, so lets drop this ;-)
> Well then, how about "ticket_dep" or "ticket_req"?

The first sounds a bit better to me.

> > Not sure the kind="Deadman" is actually required, but it probably makes
> > sense to be able to switch off the big hammer for debugging purposes.
> > ;-)
> I was thinking it's for switching on/off "immediately fence once the
> dependency is no longer satisfied".

Agreed. I was saying that this is only for debugging purposes.

> > 
> > I don't see why any resource would depend on several tickets; but I can
> > see a use case for wanting to depend on _not_ owning a ticket, similar
> > to the node attributes. And the resource would need a role, obviously.
> OK. The schema I can imagine:
> 
>   <define name="element-ticket_dep">
>     <element name="ticket_dep">
>       <attribute name="id"><data type="ID"/></attribute>
>       <choice>
>         <oneOrMore>
>           <ref name="element-resource-set"/>
>         </oneOrMore>
>         <group>
>           <attribute name="rsc"><data type="IDREF"/></attribute>
>           <optional>
>             <attribute name="rsc-role">
>               <ref name="attribute-roles"/>
>             </attribute>
>           </optional>
>         </group>
>       </choice>
>       <attribute name="ticket"><text/></attribute>

Actually, if we were to define a new type for the ticket list, we could
reference the id of the ticket element here and only allow configured
tickets.

> > No to both; it can be revoked manually, yes, but it isn't going to be
> > always the case. I'm also not quite sure I understand where this
> > question is headed; how does it matter here whether the ticket is
> > revoked manually or not?
> I was just thinking -- before we have the CTR, we rely on the admin
> quite much.

Yes, the CTR sort-of replaces (or at least augments) the administrator.


> > A site that does not own a ticket but has a non-zero value for it
> > defined will request the ticket from the CTR; the CTR will grant it to
> > the site with the highest bid (but not to a site with 0)
> The site with the highest "bid" is being revoked the ticket.

Hm? No, I thought the site with the highest bid would be granted the
ticket, not have it revoked.

> Should it clear the "bid" also? Otherwise it will get the ticket again
> soon after?

My thinking was that the "bid" is actually an on-going event/finite
state machine. Whenever a state change occurs - connection dropped
somewhere, bid value changed, a site becomes non-quorate and gives up
its ticket, etc - the CTRs that are still alive re-evaluate the grants
and decide anew where the tickets go.

> > (Tangent - ownership appears to belong to the status section; the value
> > seems belongs to the cib->ticket section(?).)
> Perhaps. Although there's no appropriate place to set a cluster-wide
> attribute in the status section so far.

Right, and it is actually not so easy, since the status section is
reconstructed from each node at times.

> Other solutions are:
> A "ticket" is not a nvpair. It is
> 
> - An object with "ownership" and "bid" attributes.
> Or:
> - A nvpair-set which includes the "ownership" and "bid" nvpairs.

Both might work. The first - an element with several attributes - might
work best, I think, since it's a bit cleaner, and would allow us to do
more validation when dependencies are defined.

Regards,
    Lars

-- 
Architect Storage/HA, OPS Engineering, Novell, Inc.
SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 21284 (AG Nürnberg)
"Experience is the name everyone gives to their mistakes." -- Oscar Wilde