[ClusterLabs] Antw: Placing resource based on least load on a node

[Michael Schwartzkopff]

Am 20.11.18 um 08:35 schrieb Bernd:
> Am 2018-11-20 08:06, schrieb Ulrich Windl:
>>>>> Bernd <bernd at kroenchenstadt.de> schrieb am 20.11.2018 um 07:21 in
>>>>> Nachricht
>> <dbae607c63168d4e14584abfba0b4b1b at kroenchenstadt.de>:
>>> Hi,
>>>
>>> I'd like to run a certain bunch of cronjobs from time to time on the
>>> cluster node (four node cluster) that has the lowest load of all four
>>> nodes.
>>>
>>> The parameters wanted for this system yet to build are
>>>
>>> * automatic placement on one of the four nodes (i.e., that with the
>>> lowest load)
>>>
>>> * in case a node fails, automatically removed from the cluster
>>>
>>> * it must only exist a single entity of the cronjob entity running
>>>
>>> so this really screams for pacemakter being used as foundation.
>>>
>>> However, I'm not sure how to implement the "put onto node with least
>>> load" part. I was thinking to use Node Attributes for that, but I
>>> didn't
>>> find any solution "out of the box" for this. Furthermore, as load is a
>>> highly volatile value, how can one make sure that all cronjobs are run
>>> to the end without being moved to a node that possibly meanwhile got a
>>> lower load than the one executing the jobs?
>>
>> Hi!
>>
>> Actually I think the last one is the easiest (assuming the cron jobs
>> do not need any resources that are moved): Once a cron job is started,
>> it will run until it ends, whether it's crontab has been moved or not.
>>
>> Despite of that I think cluster software is not ideal when you
>> actually need load-balancing software.
>>
>> Regards,
>> Ulrich
>
> The only resource(s) existing would be the cron "runner".
>
> The point about load balancing is true, yes... so, any idea what to
> use instead? Is there already a tool or framework for solving a
> problem like this available or do I have to start from scratch? Not
> that I'd be too lazy, but what's the use of reinventing the wheel
> repeatedly...? ;)
>
> Regards,
>
> Bernd
> _______________________________________________
> Users mailing list: Users at clusterlabs.org
> https://lists.clusterlabs.org/mailman/listinfo/users
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org


hi,


I solved this problem years ago. I used the utilization attribute. But
you can use any attribute. You have to write an agent that measures the
CPU load every X minutes and updates the attribute.  Now you just have
to add a location constraint, that starts the resource on the node with
the "best" attribute value. The "best" could be lowest CPU usage or most
free RAM or whatever you want.


The disadvantage of this solution is that the cluster (i.e. pacemaker)
has to recalculate the scores every time you update your attribute. That
causes additional load. If you have many resources the interdepend that
additional load may be not negligible.


Greetings,


Mit freundlichen Grüßen,

-- 

[*] sys4 AG
 
https://sys4.de, +49 (89) 30 90 46 64
Schleißheimer Straße 26/MG,80333 München
 
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer, Wolfgang Stief
Aufsichtsratsvorsitzender: Florian Kirstein


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 213 bytes
Desc: OpenPGP digital signature
URL: <https://lists.clusterlabs.org/pipermail/users/attachments/20181120/e6f26313/attachment-0002.sig>