[ClusterLabs Developers] [openstack-dev] [HA] future of OpenStack OCF resource agents (was: resource-agents v4.2.0)

Fri Nov 2 12:03:17 UTC 2018

On 01/11/18 17:07 -0500, Ken Gaillot wrote:
> FYI there is further discussion happening on the PR:
> 
> https://github.com/ClusterLabs/resource-agents/pull/1147
> 
> I think we have multiple issues we're trying to solve:
> 
> 1. Discoverability in terms of users knowing what agents may be
> available for any given purpose
> 
> 2. Organization of installed agents into categories for display by
> tools (especially GUIs)
> 
> 3. Packaging of agents in a way that avoids dragging in unnecessary
> dependencies
> 
> 4. Then the original problem that the provider field solved, which was
> to allow unrelated organizations to create their own sets of resource
> agents, whether for internal use or public distribution. This has
> become less of a problem since the consolidation of the cluster stack,
> but is still useful, especially for local customizations of stock
> agents.
> 
> #1 and #4 conflict to some extent. I think the whole point of a
> standard (OCF in this case) is to not require a central authority, so
> we should allow for independent providers who may not be closely tied
> into the ClusterLabs community. But we don't want users to be lost in
> a sea of unconnected repos, either.
> 
> I don't see an obvious solution for #1 (discoverability). We could
> document known other repos in the resource-agents README as some have
> suggested, or some other common location such as clusterlabs.org, but
> will users know to look there?

Main problem with discoverability is that with the upstream hat on
(both these are getting intertwined contentiously, though the context
matters, and only the former should be of utmost concern here), you want
to serve every and each cluster stack user, regardless of platform and
its conventions for procurement of the software bits, while making it
immediately actionable without hassles of finding intermediate manual
steps.  While on the other hand, there is a heterogenous set of
packages.

Luckily, no wheel reinventing needed, Zero Install projects looks truly
appealing in this regard: https://0install.net/

- atomic SW pieces/packages are offered by the means of XMLs
  (hence user scripting friendly) that can also be served in
  a web browser where they render as human friendly pages
  thanks to XSLT, see an example:
  http://roscidus.com/0mirror/sites/site-rox.sourceforge.net.html

- feeds can refer to particular platform-native packages
  that will be preferred when possible, and for yet unsupported
  platforms, there could be some good enough generalized recipe
  (download, prepare if needed, pick the suitable files):
  https://0install.net/distribution-integration.html#idm44

- heterogenous package sources are the core, so non-resource-agents
  agents would no longer be ostracized as long as someone cares
  to send submissions with the feeds for the external projects
  (it could easily be in the form of git-repo-backed website,
  so the workflow for such new submissions would be trivial)

Perhaps it would be a good idea to eventually put such feeds together
to be served under clusterlabs.org domain, akin to one-stop shops
("app stores") that everyone is so familiar with these days?

For that to work with enough granularity (of semi-standalone agents),
it's vital to push downstreams to start packaging agents at this
granular level (now, I am looking at resource-agents), so that the
you-only-get-what-you-ask-for use case is possible at all.

With this in place and properly advertised, I believe the problem
of where to stick and maintain particular agents would become
really minor as long as shared dependencies are stabilized.
Also, all agents would be comparably equal in the feeds' listing,
which I think would be a leap towards democratizing the landscape
(e.g., when someone is a dedicated author of some agent to scratch
her itch, while preferring an indepedent, autonomous yet publicized
development).

And indeed, rinse and repeat for any other agents (fence agents,
alert handlers), and in turn, let people combine available kit
components as they like.

> For #2 (organization), some possibilities are:
> 
> - add a category field in the RA meta-data
> - extend the RA naming to include a category, e.g.
> ocf:clusterlabs:networking/IPaddr2
> - repurpose the provider field as a category name
> 
> The first is cleaner and works unchanged with existing tools, but it
> requires any tool that wants to use it to read all agents' meta-data at
> start-up. I'm not sure if that's reasonable or not. The second allows
> more efficient listing (just regular old subdirectories) but may
> require changes to the standard as well existing tools. I'm not fond of
> the third, because it then loses the ability to solve #4. Of course I'm
> open to other possibilities too :-)
> 
> I'm not sure how much a problem #3 (packaging) is. Just because an
> agent manipulates service X doesn't mean it needs to depend on the X
> package;

Again, this is a downstream discussion not very appropriate here,
but for the "app store"/you-only-get-what-you-ask-for scenario
sketched above to combine with downstream packaging easily, it makes
damn a lot of sense to have individual agents installable in separation,
and in that case, it'd be wise for these individual (downstream)
packages to require whatever they need to be useful (for convenience,
since they are at that time known to be wanted for actual deployment,
not an unwanted cruft in the system).

[daring to run into even greater off-topic: for RPM world in
particular, rich/boolean dependencies make it quite easy for
resource-agents to retain existing catch-all + no-extra-deps
semantics while imposing detailed prerequisites when particular
agents installed individually:
  %package heartbeat-apache
  Requires: (httpd unless resource-agents-all)
]

> we could limit dependencies to the interpreters used. On the other
> hand, using application-specific Python libraries, for example,
> would reasonably be a package dependency that we wouldn't want to
> require for all installations. So, maybe the resource-agents package
> could separate into resource-agents-core (for all agents with
> minimal dependencies), and separate resource-agents-X packages for
> those with highly specific library dependencies. This would all be
> within the ClusterLabs resource-agents project.

See above.

> I like keeping the provider field for #4 (independent distributors),
> though I'm open to changing it.

It guess it would suddenly make a bigger sense in "app store"
perspective (these agents are endorsed directly by the immediate
contributors to the cluster stack, and these are rather specific
to what "X" provider has an interest in).

Hope this is a worthy perspective.

-- 
Nazdar,
Jan (Poki)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <https://lists.clusterlabs.org/pipermail/developers/attachments/20181102/b43e83f6/attachment-0003.sig>