[ClusterLabs] PCSD - High Memory Usage
tojeline at redhat.com
Thu Jun 27 09:30:52 EDT 2019
We (pcs developers) do not have any statistics regarding pcsd memory
consumption on a cluster with that many nodes and resources. So I cannot
confirm if this is normal or not. That being said, it seems to me the
memory usage is higher than it should be.
Over the time, there has been reports of pcsd consuming unreasonably
high amounts of memory. These issues are hard to reproduce and they do
not happen every time so we haven't been able to track them down and
fix. The problem may not even be in pcsd code itself, it may be buried
in rubygems, libraries or ruby. (There was a bug in ruby threads causing
pcsd using 100% CPU.)
We are working on overhauling pcsd to a new architecture while moving
its codebase from Ruby to Python. The architecture gives us more control
over pcsd worker threads which should help us lower pcsd memory
footprint. It is a long term goal, however, definitely not something
which would be ready in a few months.
For now the only advice I have is to restart pcsd from time to time when
you think its memory footprint is too high. Or, if you don't use web UI,
you can stop pcsd completely. The cluster does not depend on pcsd
running, it is only needed for managing the cluster.
I understand this is far from a solution but at this time I cannot offer
Dne 21. 06. 19 v 13:32 Daniel Brant napsal(a):
> I'm running a cluster with 9 active nodes (4 in standby), on this
> cluster are 258 registered resources. 99% of these are perl and java
> process ran via multi-target systemd units, 2 are heartbeat IPaddr2 and
> 1 fence device. I am seeing very high memory usage in the pcsd process,
> 2.5GB to 2.7GB on most node with one running 3.3GB.
> Is this kind of memory usage to be expected when managing this volume of
> resources? I am running pacemaker version 1.1.19 on CentOS7.6.1810.
> Any suggestions or advice is greatly appreciated.
> This email is from the Press Association. For more information, see
> www.pressassociation.com. This email may contain confidential
> information. Only the addressee is permitted to read, copy, distribute
> or otherwise use this email or any attachments. If you have received it
> in error, please contact the sender immediately. Any opinion expressed
> in this email is personal to the sender and may not reflect the opinion
> of the Press Association. Any email reply to this address may be subject
> to interception or monitoring for operational reasons or for lawful
> business practices.
> Manage your subscription:
> ClusterLabs home: https://www.clusterlabs.org/
More information about the Users