[ClusterLabs] Antw: crm_report consumes all available RAM

[Dejan Muhamedagic]

Hi Vladislav,

On Mon, Oct 05, 2015 at 07:00:18PM +0300, Vladislav Bogdanov wrote:
> 14.09.2015 02:31, Andrew Beekhof wrote:
> >
> >>On 8 Sep 2015, at 10:18 pm, Ulrich Windl <Ulrich.Windl at rz.uni-regensburg.de> wrote:
> >>
> >>>>>Vladislav Bogdanov <bubble at hoster-ok.com> schrieb am 08.09.2015 um 14:05 in
> >>Nachricht <55EECEFB.8050001 at hoster-ok.com>:
> >>>Hi,
> >>>
> >>>just discovered very interesting issue.
> >>>If there is a system user with very big UID (80000002 in my case),
> >>>then crm_report (actually 'grep' it runs) consumes too much RAM.
> >>>
> >>>Relevant part of the process tree at that moment looks like (word-wrap off):
> >>>USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> >>>...
> >>>root     25526  0.0  0.0 106364   636 ?        S    12:37   0:00          \_
> >>>/bin/sh /usr/sbin/crm_report --dest=/var/log/crm_report -f 0000-01-01 00:00:00
> >>>root     25585  0.0  0.0 106364   636 ?        S    12:37   0:00
> >>>\_ bash /var/log/crm_report/collector
> >>>root     25613  0.0  0.0 106364   152 ?        S    12:37   0:00
> >>>     \_ bash /var/log/crm_report/collector
> >>>root     25614  0.0  0.0 106364   692 ?        S    12:37   0:00
> >>>         \_ bash /var/log/crm_report/collector
> >>>root     27965  4.9  0.0 100936   452 ?        S    12:38   0:01
> >>>         |   \_ cat /var/log/lastlog
> >>>root     27966 23.0 82.9 3248996 1594688 ?     D    12:38   0:08
> >>>         |   \_ grep -l -e Starting Pacemaker
> >>>root     25615  0.0  0.0 155432   600 ?        S    12:37   0:00
> >>>         \_ sort -u
> >>>
> >>>ls -ls /var/log/lastlog shows:
> >>>40 -rw-r--r--. 1 root root 23360000876 Sep  8 04:36 /var/log/lastlog
> >>>
> >>>That is sparse binary file, which consumes only 40k of disk space.
> >>>At the same time its size is 23GB, and grep takes all the RAM trying to
> >>>grep a string from a 23GB of mostly zeroes without new-lines.
> >>>
> >>>I believe this is worth fixing,
> >
> >Shouldn’t this be directed to the grep folks?
> 
> Actually, not everything in /var/log are textual logs. Currently
> findmsg() [z,bz,xz]cats _every_ file there and greps for a pattern.
> Shouldn't it skip some well-known ones? btmp, lastlog and wtmp are
> good candidates to be skipped. They are not intended to be handled
> as a text.
> 
> Or may be just test that file is a text in a find_decompressor() and
> to not cat it if it is not?
> 
> something like
> find_decompressor() {
>     if echo $1 | grep -qs 'bz2$'; then
>         echo "bzip2 -dc"
>     elif echo $1 | grep -qs 'gz$'; then
>         echo "gzip -dc"
>     elif echo $1 | grep -qs 'xz$'; then
>         echo "xz -dc"
>     elif file $1 | grep -qs 'text'; then
>         echo "cat"
>     else
>         echo "echo"

Good idea.

Cheers,

Dejan

>     fi
> }
> 
> 
> >
> >
> >_______________________________________________
> >Users mailing list: Users at clusterlabs.org
> >http://clusterlabs.org/mailman/listinfo/users
> >
> >Project Home: http://www.clusterlabs.org
> >Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> >Bugs: http://bugs.clusterlabs.org
> >
> 
> 
> _______________________________________________
> Users mailing list: Users at clusterlabs.org
> http://clusterlabs.org/mailman/listinfo/users
> 
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org