[ClusterLabs] Idea of native masking of systemd resources (Was: Configure a resource to only run a single instance at all times)

[Jan Pokorný]

On 29/10/18 20:19 +0300, Andrei Borzenkov wrote:
> 29.10.2018 20:04, jm2109384 at gmail.com пишет:
>> I'm a new user of pacemaker clustering software and I've just configured a
>> cluster with a single systemd resource. I have the following cluster and
>> resource configurations below. Failover works perfectly between the two
>> nodes however, i wanted to have a constraint/rule or a config that will
>> ensure that my resource has a single instance running on the cluster at all
>> times. I'd like to avoid the situation where the resource gets started
>> manually and ends up running on both cluster nodes.  Hoping to get your
>> advice on how to achieve this. Thanks in advance.
> 
> pacemaker does one time probe on each node when pacemaker is started.
> This covers the case when resource was manually started before
> pacemaker. You can enable monitor on inactive nodes which should also
> detect if resource was started outside of pacemaker. But note that it
> leaves you some window (up to monitoring interval) when multiple
> instances may be up on different nodes until pacemaker is aware of it.

Exactly; it would have to be up to the configuration possibilities
of your platform/OS to possibly lock any other source of initial
kick off of the resource at hand down.

Speaking of systemd in particular, it provides it's own gating
mechanism that may come handy in cases where a risk of resource
getting started outside systemd's control is negligible
(e.g. the startup sequence is so complex nobody will ever
want to emulate existing systemd arrangement by external
means) ... "systemctl mask <unit>":
https://www.freedesktop.org/software/systemd/man/systemctl.html#mask%20UNIT%E2%80%A6

It could even be meaningfully integrated into pacemaker configuration,
just do something rather hackish (and untested, really just a sketch
to demonstrate an idea) like:

  [...]
  pcs -f in-prep.xml resource create foo systemd:foo.service
  pcs -f in-prep.xml resource create foo-disabler \
    ocf:heartbeat:anything binfile=/usr/bin/watch \
    "cmdline_options=-n999 -- systemctl mask --runtime foo.service"
  pcs -f in-prep.xml constraint colocation \
    set foo-disabler set foo role=Stopped
  [...]

Here, the intention is to keep foo.service assurably masked in systemd
whenever it is supposed to be stopped at given machine.

It's up for discussion whether it would be a good idea if pacemaker
actually did something to that effect natively, while behaving
reasonably with unmanaged resources, not ever blocking its own way
with this masking, etc.  There would be some gains, but at certain
costs stemming from internal and external (documentation) complexity.

-- 
Nazdar,
Jan (Poki)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20181031/f1773777/attachment-0002.sig>