[ClusterLabs] VirtualDomain and Resource_is_Too_Active ?? - problem/error

Wed May 29 06:42:37 EDT 2019

hi guys,

I doing something which I believe is fairly simple, namely:

$ pcs resource create HA-work9-win10-kvm VirtualDomain
hypervisor="qemu:///system"
config="/0-ALL.SYSDATA/QEMU_VMs/HA-work9-win10.qcow2"
migration_transport=ssh --disable

virt guest is good, runs in libvirth okey, yet pacemaker fails:

...

  notice: State transition S_IDLE -> S_POLICY_ENGINE
   error: Invalid recurring action
chenbro0.1-raid5-mnt-start-interval-90 wth name: 'start'
   error: Invalid recurring action chenbro0.1-raid5-mnt-stop-interval-90
wth name: 'stop'
  notice: Calculated transition 1864, saving inputs in
/var/lib/pacemaker/pengine/pe-input-2022.bz2
  notice: Configuration ERRORs found during PE processing.  Please run
"crm_verify -L" to identify issues.
  notice: Initiating monitor operation HA-work9-win10-kvm_monitor_0
locally on whale.private
  notice: Initiating monitor operation HA-work9-win10-kvm_monitor_0 on
swir.private
  notice: Initiating monitor operation HA-work9-win10-kvm_monitor_0 on
rider.private
 warning: HA-work9-win10-kvm_monitor_0 process (PID 2103512) timed out
 warning: HA-work9-win10-kvm_monitor_0:2103512 - timed out after 30000ms
  notice: HA-work9-win10-kvm_monitor_0:2103512:stderr [
/usr/lib/ocf/resource.d/heartbeat/VirtualDomain: line 981: [: too many
arguments ]
   error: Result of probe operation for HA-work9-win10-kvm on
whale.private: Timed Out
  notice: whale.private-HA-work9-win10-kvm_monitor_0:204 [
/usr/lib/ocf/resource.d/heartbeat/VirtualDomain: line 981: [: too many
arguments\n ]
 warning: Action 15 (HA-work9-win10-kvm_monitor_0) on rider.private
failed (target: 7 vs. rc: 1): Error
  notice: Transition aborted by operation HA-work9-win10-kvm_monitor_0
'modify' on rider.private: Event failed
 warning: Action 17 (HA-work9-win10-kvm_monitor_0) on whale.private
failed (target: 7 vs. rc: 1): Error
 warning: Action 16 (HA-work9-win10-kvm_monitor_0) on swir.private
failed (target: 7 vs. rc: 1): Error
  notice: Transition 1864 (Complete=3, Pending=0, Fired=0, Skipped=0,
Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-2022.bz2):
Complete
 warning: Processing failed probe of HA-work9-win10-kvm on
whale.private: unknown error
  notice: If it is not possible for HA-work9-win10-kvm to run on
whale.private, see the resource-discovery option for location constraints
 warning: Processing failed probe of HA-work9-win10-kvm on
whale.private: unknown error
  notice: If it is not possible for HA-work9-win10-kvm to run on
whale.private, see the resource-discovery option for location constraints
 warning: Processing failed probe of HA-work9-win10-kvm on swir.private:
unknown error
  notice: If it is not possible for HA-work9-win10-kvm to run on
swir.private, see the resource-discovery option for location constraints
 warning: Processing failed probe of HA-work9-win10-kvm on swir.private:
unknown error
  notice: If it is not possible for HA-work9-win10-kvm to run on
swir.private, see the resource-discovery option for location constraints
 warning: Processing failed probe of HA-work9-win10-kvm on
rider.private: unknown error
  notice: If it is not possible for HA-work9-win10-kvm to run on
rider.private, see the resource-discovery option for location constraints
 warning: Processing failed probe of HA-work9-win10-kvm on
rider.private: unknown error
  notice: If it is not possible for HA-work9-win10-kvm to run on
rider.private, see the resource-discovery option for location constraints
   error: Invalid recurring action
chenbro0.1-raid5-mnt-start-interval-90 wth name: 'start'
   error: Invalid recurring action chenbro0.1-raid5-mnt-stop-interval-90
wth name: 'stop'
   error: Resource HA-work9-win10-kvm is active on 3 nodes (attempting
recovery)
  notice: See
https://wiki.clusterlabs.org/wiki/FAQ#Resource_is_Too_Active for more
information
  notice:  * Stop       HA-work9-win10-kvm      
(                                             whale.private )   due to
node availability
  notice:  * Stop       HA-work9-win10-kvm      
(                                              swir.private )   due to
node availability
  notice:  * Stop       HA-work9-win10-kvm      
(                                             rider.private )   due to
node availability
   error: Calculated transition 1865 (with errors), saving inputs in
/var/lib/pacemaker/pengine/pe-error-56.bz2
  notice: Configuration ERRORs found during PE processing.  Please run
"crm_verify -L" to identify issues.
  notice: Initiating stop operation HA-work9-win10-kvm_stop_0 on
rider.private
  notice: Initiating stop operation HA-work9-win10-kvm_stop_0 on
swir.private
  notice: Initiating stop operation HA-work9-win10-kvm_stop_0 locally on
whale.private
 warning: Action 17 (HA-work9-win10-kvm_stop_0) on rider.private failed
(target: 0 vs. rc: 1): Error
  notice: Transition aborted by operation HA-work9-win10-kvm_stop_0
'modify' on rider.private: Event failed
  notice: Transition aborted by
status-3-fail-count-HA-work9-win10-kvm.stop_0 doing create
fail-count-HA-work9-win10-kvm#stop_0=INFINITY: Transient attribute change
  notice: HA-work9-win10-kvm_stop_0:2104195:stderr [
/usr/lib/ocf/resource.d/heartbeat/VirtualDomain: line 981: [: too many
arguments ]
  notice: HA-work9-win10-kvm_stop_0:2104195:stderr [ error: unexpected
data 'file' ]
  notice: HA-work9-win10-kvm_stop_0:2104195:stderr [
ocf-exit-reason:forced stop failed ]
  notice: Result of stop operation for HA-work9-win10-kvm on
whale.private: 1 (unknown error)
  notice: whale.private-HA-work9-win10-kvm_stop_0:205 [
/usr/lib/ocf/resource.d/heartbeat/VirtualDomain: line 981: [: too many
arguments\nerror: unexpected data 'file'\nocf-exit-reason:forced stop
failed\n ]
 warning: Action 13 (HA-work9-win10-kvm_stop_0) on whale.private failed
(target: 0 vs. rc: 1): Error
 warning: Action 14 (HA-work9-win10-kvm_stop_0) on swir.private failed
(target: 0 vs. rc: 1): Error
  notice: Transition 1865 (Complete=3, Pending=0, Fired=0, Skipped=0,
Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-error-56.bz2): Complete
 warning: Processing failed stop of HA-work9-win10-kvm on whale.private:
unknown error
 warning: Processing failed stop of HA-work9-win10-kvm on whale.private:
unknown error
 warning: Processing failed stop of HA-work9-win10-kvm on swir.private:
unknown error
 warning: Processing failed stop of HA-work9-win10-kvm on swir.private:
unknown error
 warning: Processing failed stop of HA-work9-win10-kvm on rider.private:
unknown error
 warning: Processing failed stop of HA-work9-win10-kvm on rider.private:
unknown error
 warning: Forcing HA-work9-win10-kvm away from rider.private after
1000000 failures (max=1000000)
 warning: Forcing HA-work9-win10-kvm away from whale.private after
1000000 failures (max=1000000)
   error: Invalid recurring action
chenbro0.1-raid5-mnt-start-interval-90 wth name: 'start'
   error: Invalid recurring action chenbro0.1-raid5-mnt-stop-interval-90
wth name: 'stop'
   error: Resource HA-work9-win10-kvm is active on 3 nodes (attempting
recovery)
  notice: See
https://wiki.clusterlabs.org/wiki/FAQ#Resource_is_Too_Active for more
information
   error: Calculated transition 1866 (with errors), saving inputs in
/var/lib/pacemaker/pengine/pe-error-57.bz2
  notice: Configuration ERRORs found during PE processing.  Please run
"crm_verify -L" to identify issues.
 warning: Processing failed stop of HA-work9-win10-kvm on whale.private:
unknown error
 warning: Processing failed stop of HA-work9-win10-kvm on whale.private:
unknown error
 warning: Processing failed stop of HA-work9-win10-kvm on swir.private:
unknown error
 warning: Processing failed stop of HA-work9-win10-kvm on swir.private:
unknown error
 warning: Processing failed stop of HA-work9-win10-kvm on rider.private:
unknown error
 warning: Processing failed stop of HA-work9-win10-kvm on rider.private:
unknown error
 warning: Forcing HA-work9-win10-kvm away from rider.private after
1000000 failures (max=1000000)
 warning: Forcing HA-work9-win10-kvm away from swir.private after
1000000 failures (max=1000000)
 warning: Forcing HA-work9-win10-kvm away from whale.private after
1000000 failures (max=1000000)
   error: Invalid recurring action
chenbro0.1-raid5-mnt-start-interval-90 wth name: 'start'
   error: Invalid recurring action chenbro0.1-raid5-mnt-stop-interval-90
wth name: 'stop'
   error: Resource HA-work9-win10-kvm is active on 3 nodes (attempting
recovery)
  notice: See
https://wiki.clusterlabs.org/wiki/FAQ#Resource_is_Too_Active for more
information
   error: Calculated transition 1867 (with errors), saving inputs in
/var/lib/pacemaker/pengine/pe-error-58.bz2
  notice: Transition 1867 (Complete=0, Pending=0, Fired=0, Skipped=0,
Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-error-58.bz2): Complete
  notice: Configuration ERRORs found during PE processing.  Please run
"crm_verify -L" to identify issues.
...

$ pcs status --all

...

Failed Actions:
* HA-work9-win10-kvm_stop_0 on whale.private 'unknown error' (1):
call=205, status=complete, exitreason='forced stop failed',
    last-rc-change='Wed May 29 11:32:23 2019', queued=0ms, exec=3158ms
* HA-work9-win10-kvm_stop_0 on swir.private 'unknown error' (1):
call=125, status=complete, exitreason='forced stop failed',
    last-rc-change='Wed May 29 11:32:23 2019', queued=0ms, exec=3398ms
* HA-work9-win10-kvm_stop_0 on rider.private 'unknown error' (1):
call=129, status=complete, exitreason='forced stop failed',
    last-rc-change='Wed May 29 11:32:23 2019', queued=0ms, exec=2934ms

$ crm_verify -L -V
   error: unpack_rsc_op:    Preventing HA-work9-win10-kvm from
re-starting anywhere: operation stop failed 'not configured' (6)
   error: unpack_rsc_op:    Preventing HA-work9-win10-kvm from
re-starting anywhere: operation stop failed 'not configured' (6)
   error: unpack_rsc_op:    Preventing HA-work9-win10-kvm from
re-starting anywhere: operation stop failed 'not configured' (6)
   error: unpack_rsc_op:    Preventing HA-work9-win10-kvm from
re-starting anywhere: operation stop failed 'not configured' (6)
   error: unpack_rsc_op:    Preventing HA-work9-win10-kvm from
re-starting anywhere: operation stop failed 'not configured' (6)
   error: unpack_rsc_op:    Preventing HA-work9-win10-kvm from
re-starting anywhere: operation stop failed 'not configured' (6)
   error: native_create_actions:    Resource HA-work9-win10-kvm is
active on 3 nodes (attempting recovery)

Something buggy there, or I'm missing something obvious?

many thanks, L.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: pEpkey.asc
Type: application/pgp-keys
Size: 1757 bytes
Desc: not available
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20190529/499271c6/attachment-0001.bin>