[ClusterLabs] Antw: Placing resource based on least load on a node

[Bernd]

Am 2018-11-20 08:41, schrieb Michael Schwartzkopff:
> Am 20.11.18 um 08:35 schrieb Bernd:
>> Am 2018-11-20 08:06, schrieb Ulrich Windl:
>>>>>> Bernd <bernd at kroenchenstadt.de> schrieb am 20.11.2018 um 07:21 in
>>>>>> Nachricht
>>> <dbae607c63168d4e14584abfba0b4b1b at kroenchenstadt.de>:
>>>> Hi,
>>>> 
>>>> I'd like to run a certain bunch of cronjobs from time to time on the
>>>> cluster node (four node cluster) that has the lowest load of all 
>>>> four
>>>> nodes.
>>>> 
>>>> The parameters wanted for this system yet to build are
>>>> 
>>>> * automatic placement on one of the four nodes (i.e., that with the
>>>> lowest load)
>>>> 
>>>> * in case a node fails, automatically removed from the cluster
>>>> 
>>>> * it must only exist a single entity of the cronjob entity running
>>>> 
>>>> so this really screams for pacemakter being used as foundation.
>>>> 
>>>> However, I'm not sure how to implement the "put onto node with least
>>>> load" part. I was thinking to use Node Attributes for that, but I
>>>> didn't
>>>> find any solution "out of the box" for this. Furthermore, as load is 
>>>> a
>>>> highly volatile value, how can one make sure that all cronjobs are 
>>>> run
>>>> to the end without being moved to a node that possibly meanwhile got 
>>>> a
>>>> lower load than the one executing the jobs?
>>> 
>>> Hi!
>>> 
>>> Actually I think the last one is the easiest (assuming the cron jobs
>>> do not need any resources that are moved): Once a cron job is 
>>> started,
>>> it will run until it ends, whether it's crontab has been moved or 
>>> not.
>>> 
>>> Despite of that I think cluster software is not ideal when you
>>> actually need load-balancing software.
>>> 
>>> Regards,
>>> Ulrich
>> 
>> The only resource(s) existing would be the cron "runner".
>> 
>> The point about load balancing is true, yes... so, any idea what to
>> use instead? Is there already a tool or framework for solving a
>> problem like this available or do I have to start from scratch? Not
>> that I'd be too lazy, but what's the use of reinventing the wheel
>> repeatedly...? ;)
>> 
>> Regards,
>> 
>> Bernd
>> _______________________________________________
>> Users mailing list: Users at clusterlabs.org
>> https://lists.clusterlabs.org/mailman/listinfo/users
>> 
>> Project Home: http://www.clusterlabs.org
>> Getting started: 
>> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> Bugs: http://bugs.clusterlabs.org
> 
> 
> hi,
> 
> 
> I solved this problem years ago. I used the utilization attribute. But
> you can use any attribute. You have to write an agent that measures the
> CPU load every X minutes and updates the attribute.  Now you just have
> to add a location constraint, that starts the resource on the node with
> the "best" attribute value. The "best" could be lowest CPU usage or 
> most
> free RAM or whatever you want.
> 
> 
> The disadvantage of this solution is that the cluster (i.e. pacemaker)
> has to recalculate the scores every time you update your attribute. 
> That
> causes additional load. If you have many resources the interdepend that
> additional load may be not negligible.
> 
> 
> Greetings,

Hi,

yes, that was my idea, too, see original post: Gathering the individual 
nodes' load, and use Node Attributes (utilization) to help crmd to 
decide where to put stuff.

As this would be the only resource of the cluster (all nodes otherwise 
run a shared nothing approach) load would be no issue.

I built something like that in 2013, IIRC, but it was not pure cron – we 
wrote a kind of wrapper (runner) as resource. That was a bit different.

Thanks for sharing!

Regards,

Bernd