[ClusterLabs] Antw: Re: PCSD - High Memory Usage

[Ulrich Windl]

Would running pcsd unter valgrind be an option? In addition to checking for
leaks, it can also provide some memory usage statistics (who is using how
much)...

>>> Tomas Jelinek <tojeline at redhat.com> schrieb am 27.06.2019 um 15:30 in
Nachricht
<363f827e-d05d-309f-7ab6-c43e268df8fd at redhat.com>:
> Hi,
> 
> We (pcs developers) do not have any statistics regarding pcsd memory 
> consumption on a cluster with that many nodes and resources. So I cannot 
> confirm if this is normal or not. That being said, it seems to me the 
> memory usage is higher than it should be.
> 
> Over the time, there has been reports of pcsd consuming unreasonably 
> high amounts of memory. These issues are hard to reproduce and they do 
> not happen every time so we haven't been able to track them down and 
> fix. The problem may not even be in pcsd code itself, it may be buried 
> in rubygems, libraries or ruby. (There was a bug in ruby threads causing 
> pcsd using 100% CPU.)
> 
> We are working on overhauling pcsd to a new architecture while moving 
> its codebase from Ruby to Python. The architecture gives us more control 
> over pcsd worker threads which should help us lower pcsd memory 
> footprint. It is a long term goal, however, definitely not something 
> which would be ready in a few months.
> 
> For now the only advice I have is to restart pcsd from time to time when 
> you think its memory footprint is too high. Or, if you don't use web UI, 
> you can stop pcsd completely. The cluster does not depend on pcsd 
> running, it is only needed for managing the cluster.
> 
> I understand this is far from a solution but at this time I cannot offer 
> anything else.
> 
> 
> Tomas
> 
> 
> Dne 21. 06. 19 v 13:32 Daniel Brant napsal(a):
>> Hi,
>> 
>> I'm running a cluster with 9 active nodes (4 in standby), on this 
>> cluster are 258 registered resources. 99% of these are perl and java 
>> process ran via multi‑target systemd units, 2 are heartbeat IPaddr2 and 
>> 1 fence device. I am seeing very high memory usage in the pcsd process, 
>> 2.5GB to 2.7GB on most node with one running 3.3GB.
>> 
>> Is this kind of memory usage to be expected when managing this volume of 
>> resources? I am running pacemaker version 1.1.19 on CentOS7.6.1810.
>> 
>> Any suggestions or advice is greatly appreciated.
>> 
>> Thanks,
>> 
>> Danny
>> 
>> 
>> This email is from the Press Association. For more information, see 
>> www.pressassociation.com. This email may contain confidential 
>> information. Only the addressee is permitted to read, copy, distribute 
>> or otherwise use this email or any attachments. If you have received it 
>> in error, please contact the sender immediately. Any opinion expressed 
>> in this email is personal to the sender and may not reflect the opinion 
>> of the Press Association. Any email reply to this address may be subject 
>> to interception or monitoring for operational reasons or for lawful 
>> business practices.
>> 
>> _______________________________________________
>> Manage your subscription:
>> https://lists.clusterlabs.org/mailman/listinfo/users 
>> 
>> ClusterLabs home: https://www.clusterlabs.org/ 
>> 
> _______________________________________________
> Manage your subscription:
> https://lists.clusterlabs.org/mailman/listinfo/users 
> 
> ClusterLabs home: https://www.clusterlabs.org/