[ClusterLabs] Antw: Re: Antw: About globally unique resource instances distribution per node

[Ulrich Windl]

>>> Daniel Hernández <danyboy1104 at gmail.com> schrieb am 30.12.2015 um 19:43 in
Nachricht
<CAMhkz_DVOvO=SOrCDKTqERFDwoVfAWDd6=WfHvggXLKCkksUwQ at mail.gmail.com>:
> On 12/30/15, Ulrich Windl <Ulrich.Windl at rz.uni-regensburg.de> wrote:
>> Hi!
>>
>> I would expect if you set the cpu utilization per primitive (used in
clone)
>> to
>> one and set the cpu capacity per node to the correct number that no node
has
>> more primitives than the cpu number allows and that primitives are
>> distributed
>> among all available nodes. Isn't that true in your case?
>>
>> What exactly does not work in your opinion?
>>
>> Regards,
>> Ulrich
>>
>>>>> Daniel Hernández <danyboy1104 at gmail.com> schrieb am 29.12.2015 um 16:21
>> in
>> Nachricht
>> <CAMhkz_BUEjidSkeJ7uJrTJ1v-vkA+s2YWR69PF6=gOwrOSFwUg at mail.gmail.com>:
>>> Good day, I work at Datys Soluciones Tecnológicas, we use Corosync and
>>> Pacemaker in production to run a service infrastructure since 3 years
>>> ago. Versions
>>> used are Centos 6.3, corosync 1.4.1 and pacemaker 1.1.7. We have a web
>>> server, gearman
>>> job manager, and globally unique resource clones as gearman workers to
>>> balance the load distributed by gearman. My question is if there exist
>>> a way or a workaround to configure globally unique resource clone
>>> number of instances to start per node. As example: Say have 3 nodes:
>>> node1, node2 and node3, and a globally unique resource clone of 6
>>> instances with name clone_example and want to start 1 instance on
>>> node1, 2 instances on node2 and 3 instances on node3, as the following
>>> example shows.
>>>
>>> Clone Set: clone_example [example] (unique)
>>>          example:0 (ocf:heartbeat:example): Started nodo3
>>>          example:1 (ocf:heartbeat:example): Started nodo2
>>>          example:2 (ocf:heartbeat:example): Started nodo2
>>>          example:3 (ocf:heartbeat:example): Started nodo1
>>>          example:4 (ocf:heartbeat:example): Started nodo3
>>>          example:5 (ocf:heartbeat:example): Started nodo3
>>>
>>> The reason we want to configure the resource this way is because one
>>> resource clone instance consume one node cpu, and the nodes have
>>> different number of cpu:
>>> node1 = 1cpu, node2 = 2cpu, node3 = 3cpu in the example.
>>>
>>> I read Pacemaker Cluster from Scratch and Cluster Configuration
>>> Explained to find a way and see Chapter 11. Utilization and Placement
>>> Strategy. I make a test with clones and resources but the clones where
>>> not distributed as I expected and some instances were not started, I
>>> used the 3 placement-strategies and similar behaviour. I know the
>>> cluster use a
>>> best effort algorithm to distribute the resources when this option is
>>> used, and maybe that's the reason, so I am searching for a way to do
>>> it. I browse the mailing list archives to see if there exist a similar
>>> post on this topic and couldn't find it, maybe I miss it. Any response
>>> will be appreciated.
>>> Thanks for your time
>>>
>>> _______________________________________________
>>> Users mailing list: Users at clusterlabs.org 
>>> http://clusterlabs.org/mailman/listinfo/users 
>>>
>>> Project Home: http://www.clusterlabs.org 
>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf 
>>> Bugs: http://bugs.clusterlabs.org 
>>
>>
>>
>>
>> _______________________________________________
>> Users mailing list: Users at clusterlabs.org 
>> http://clusterlabs.org/mailman/listinfo/users 
>>
>> Project Home: http://www.clusterlabs.org 
>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf 
>> Bugs: http://bugs.clusterlabs.org 
>>
> 
> Hi Ulrich, thanks for your response. I take your sugestion and test my 
> example.
> I create a 3 node cluster with one dummy resource, use the following 
> commands.
> 
> crm configure primitive example1 ocf:heartbeat:Dummy \
> op monitor interval=30s
> 
> crm configure clone clone_example1 example1 \
> 	meta globally-unique="true" clone-max="6" clone-node-max="6"
> 
> 
> crm node utilization node1 set cpu 1
> crm node utilization node2 set cpu 2
> crm node utilization node3 set cpu 3
> crm resource utilization example1 set cpu 1
> crm configure property placement-strategy="balanced"
> 
> Online: [ node1 node2 node3 ]
> 
>  Clone Set: clone_example1 [example] (unique)
>      example1:0	(ocf::heartbeat:Dummy):	Started node1
>      example1:1	(ocf::heartbeat:Dummy):	Started node2
>      example1:2	(ocf::heartbeat:Dummy):	Started node3
>      example1:3	(ocf::heartbeat:Dummy):	Started node3
>      example1:4	(ocf::heartbeat:Dummy):	Started node2
>      example1:5	(ocf::heartbeat:Dummy):	Started node3
> 
> That work and was no expected by me, it work different on another
> scenario and is really why my question arrived.
> I tested an scenario in which I want to run 4 instances of resource
> example1 on node1, 3 instances on node2, 5 instances on node 3. The
> cpu capacity per node is, on node1 6, on node2 9, on node3 8, that
> because I will have other resources besides example1.
> 
> With the following cluster configuration:
> 
> crm node utilization node1 set cpu 6
> crm node utilization node2 set cpu 9
> crm node utilization node3 set cpu 8
> 
> crm resource meta clone_example1 set clone-max 12
> crm resource meta clone_example1 set clone-node-max 12
> 
> The result in the cluster is:
> Online: [ node1 node2 node3 ]
> 
>  Clone Set: clone_example1 [example1] (unique)
>      example1:0	(ocf::heartbeat:Dummy):	Started node3
>      example1:1	(ocf::heartbeat:Dummy):	Started node2
>      example1:2	(ocf::heartbeat:Dummy):	Started node3
>      example1:3	(ocf::heartbeat:Dummy):	Started node1
>      example1:4	(ocf::heartbeat:Dummy):	Started node2
>      example1:5	(ocf::heartbeat:Dummy):	Started node3
>      example1:6	(ocf::heartbeat:Dummy):	Started node2
>      example1:7	(ocf::heartbeat:Dummy):	Started node2
>      example1:8	(ocf::heartbeat:Dummy):	Started node1
>      example1:9	(ocf::heartbeat:Dummy):	Started node3
>      example1:10	(ocf::heartbeat:Dummy):	Started node2
>      example1:11	(ocf::heartbeat:Dummy):	Started node1
> 
> The cluster start 3 instances of example1 on node1 and not 4 as I
> want. That happen when I have more than 1 resource to allocate. I also
> notice that I am not setting the cluster to start 3 instances of
> example1 on node1, Is there any way to do it ?

Hi!

Sorry for the late reply, but if you want to prefer a node to get more
primitives that another, you could try to add a location constraint for those
nodes with most cpus. I haven't tried that, but that should work.

Regards,
Ulrich

> 
> _______________________________________________
> Users mailing list: Users at clusterlabs.org 
> http://clusterlabs.org/mailman/listinfo/users 
> 
> Project Home: http://www.clusterlabs.org 
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf 
> Bugs: http://bugs.clusterlabs.org