[ClusterLabs] Antw: Re: Resources start serial, not parralel
Oleg Ilyin
on.ilyin at gmail.com
Mon Dec 14 08:40:19 UTC 2015
Hi Ulrich,
thank you for your answer,
what about which limits do you talk?
The pacemaker software runs by root user, so limits should be increased for
root.
There is output from one of my server:
-bash-4.1# id
uid=0(root) gid=0(root) groups=0(root)
-bash-4.1# ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 124801
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 4096
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 124801
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
-bash-4.1# ps -ef |grep pacemaker
root 17947 1 0 Dec11 ? 00:00:09 pacemakerd
189 17953 17947 0 Dec11 ? 00:00:11 /usr/libexec/pacemaker/cib
root 17954 17947 0 Dec11 ? 00:00:14
/usr/libexec/pacemaker/stonithd
root 17955 17947 0 Dec11 ? 00:00:11 /usr/libexec/pacemaker/lrmd
189 17956 17947 0 Dec11 ? 00:00:09 /usr/libexec/pacemaker/attrd
189 17957 17947 0 Dec11 ? 00:00:09
/usr/libexec/pacemaker/pengine
root 17958 17947 0 Dec11 ? 00:00:16 /usr/libexec/pacemaker/crmd
With settings in pacemaker or system can be changed for start heavy
application ( java ) at the same time?
2015-12-14 16:29 GMT+09:00 Ulrich Windl <Ulrich.Windl at rz.uni-regensburg.de>:
> Hi!
>
> There is one feature in Linux that may affect you: If processes block on
> I/O (NFS also), the load increases, and the load is the _sum_, and not the
> _average_ of all CPUs. So if you have many CPUs, your abservable load will
> typically increase. Recently we had a load of 60, but nobody actually
> noticed ;-)
>
> So maybe you just need to adjust the limits for pacemaker...
>
> Regards,
> Ulrich
>
> >>> Oleg Ilyin <on.ilyin at gmail.com> schrieb am 13.12.2015 um 15:00 in
> Nachricht
> <CANLWoXrpvLAbcgCOE073iAOJz3zL7RAUB4UQC=0iPfGyTvnidw at mail.gmail.com>:
> > There are errors in my /var/log/messages
> >
> >
> > grep -e crmd\\[ -e crmd: /var/log/messages
> > Dec 13 00:01:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 3.290000
> > Dec 13 00:01:39 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 4.430000
> > Dec 13 00:02:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 3.900000
> > Dec 13 00:02:39 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 3.810000
> > Dec 13 00:04:25 server_name_1 crmd[9941]: notice: do_state_transition:
> > State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_
> > TIMER_POPPED origin=crm_timer_popped ]
> > Dec 13 00:04:26 server_name_1 crmd[9941]: notice: run_graph: Transition
> > 185166 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Sou
> > rce=/var/lib/pacemaker/pengine/pe-input-656.bz2):
> > Complete
> > Dec 13 00:04:26 server_name_1 crmd[9941]: notice: do_state_transition:
> > State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS c
> > ause=C_FSA_INTERNAL origin=notify_crmd ]
> > Dec 13 00:08:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 3.480000
> > Dec 13 00:09:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 5.820000
> > Dec 13 00:09:39 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 4.130000
> > Dec 13 00:10:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 4.650000
> > Dec 13 00:10:39 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 6.000000
> > Dec 13 00:11:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 5.270000
> > Dec 13 00:11:39 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 4.260000
> > Dec 13 00:12:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 3.600000
> > Dec 13 00:12:39 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 3.970000
> > Dec 13 00:13:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 3.930000
> > Dec 13 00:14:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 3.590000
> > Dec 13 00:17:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 3.350000
> > Dec 13 00:18:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 3.220000
> > Dec 13 00:18:39 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 3.830000
> > Dec 13 00:19:26 server_name_1 crmd[9941]: notice: do_state_transition:
> > State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_
> > TIMER_POPPED origin=crm_timer_popped ]
> > Dec 13 00:19:26 server_name_1 crmd[9941]: notice: run_graph: Transition
> > 185167 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Sou
> > rce=/var/lib/pacemaker/pengine/pe-input-656.bz2):
> > Complete
> > Dec 13 00:19:26 server_name_1 crmd[9941]: notice: do_state_transition:
> > State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS c
> > ause=C_FSA_INTERNAL origin=notify_crmd ]
> > Dec 13 00:24:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 3.770000
> > Dec 13 00:24:39 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 3.360000
> > Dec 13 00:25:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 3.790000
> > Dec 13 00:26:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 3.470000
> > Dec 13 00:27:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 4.890000
> > Dec 13 00:27:39 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 3.290000
> > Dec 13 00:28:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 4.040000
> > Dec 13 00:29:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 3.340000
> > Dec 13 00:29:39 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 3.250000
> > Dec 13 00:30:39 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 3.310000
> > Dec 13 00:31:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 4.880000
> > Dec 13 00:31:39 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 3.890000
> > Dec 13 00:32:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 5.320000
> > Dec 13 00:32:39 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 4.380000
> > Dec 13 00:33:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 4.430000
> > Dec 13 00:33:39 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 3.480000
> > Dec 13 00:34:26 server_name_1 crmd[9941]: notice: do_state_transition:
> > State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_
> > TIMER_POPPED origin=crm_timer_popped ]
> > Dec 13 00:34:26 server_name_1 crmd[9941]: notice: run_graph: Transition
> > 185168 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Sou
> > rce=/var/lib/pacemaker/pengine/pe-input-656.bz2):
> > Complete
> > Dec 13 00:34:26 server_name_1 crmd[9941]: notice: do_state_transition:
> > State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS c
> > ause=C_FSA_INTERNAL origin=notify_crmd ]
> > Dec 13 00:34:39 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 3.580000
> > Dec 13 00:35:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 3.350000
> > Dec 13 00:37:39 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 3.380000
> > Dec 13 00:38:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 4.400000
> > Dec 13 00:38:39 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 5.020000
> > Dec 13 00:39:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 4.080000
> > Dec 13 00:40:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 3.410000
> > Dec 13 00:40:39 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 3.490000
> > Dec 13 00:41:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 4.410000
> > Dec 13 00:41:39 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 5.120000
> > Dec 13 00:42:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 8.120000
> > Dec 13 00:42:39 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 9.040000
> > Dec 13 00:43:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 6.900000
> > Dec 13 00:43:39 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 5.830000
> > Dec 13 00:44:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 6.280000
> > Dec 13 00:44:39 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 4.540000
> > Dec 13 00:45:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 5.050000
> > Dec 13 00:45:39 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 4.170000
> > Dec 13 00:46:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 4.930000
> > Dec 13 00:46:39 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 3.520000
> > Dec 13 00:47:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 5.680000
> > Dec 13 00:47:39 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 5.050000
> > Dec 13 00:48:09 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 4.750000
> > Dec 13 00:48:39 server_name_1 crmd[9941]: notice: throttle_handle_load:
> > High CPU load detected: 3.370000
> > Dec 13 00:49:26 server_name_1 crmd[9941]: notice: do_state_transition:
> > State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_
> > TIMER_POPPED origin=crm_timer_popped ]
> > Dec 13 00:49:26 server_name_1 crmd[9941]: notice: run_graph: Transition
> > 185169 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Sou
> > rce=/var/lib/pacemaker/pengine/pe-input-656.bz2):
> > Complete
> > Dec 13 00:49:26 server_name_1 crmd[9941]: notice: do_state_transition:
> > State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS c
> > ause=C_FSA_INTERNAL origin=notify_crmd ]
> >
> >
> > 2015-12-13 21:11 GMT+09:00 Oleg Ilyin <on.ilyin at gmail.com>:
> >
> >> Hello,
> >> could you help please with issue on my environment
> >> RHEL 6.6
> >> corosync-1.4.7
> >> pacemaker-1.1.12
> >>
> >> When I start several resources in the same time by " pcs resource
> enable "
> >> command,
> >> resource doesn't start parralel, they start serial.
> >>
> >> If I start resources by one, thay start parralel. I didn't do any
> special
> >> settings.
> >>
> >> Please,
> >> which parameters can affect to this behavior of cluster?
> >>
>
>
>
>
> _______________________________________________
> Users mailing list: Users at clusterlabs.org
> http://clusterlabs.org/mailman/listinfo/users
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.clusterlabs.org/pipermail/users/attachments/20151214/f37a7f79/attachment-0002.html>
More information about the Users
mailing list