[ClusterLabs] Antw: crm_report consumes all available RAM

[Vladislav Bogdanov]

08.09.2015 15:18, Ulrich Windl wrote:
>>>> Vladislav Bogdanov <bubble at hoster-ok.com> schrieb am 08.09.2015 um 14:05 in
> Nachricht <55EECEFB.8050001 at hoster-ok.com>:
>> Hi,
>>
>> just discovered very interesting issue.
>> If there is a system user with very big UID (80000002 in my case),
>> then crm_report (actually 'grep' it runs) consumes too much RAM.
>>
>> Relevant part of the process tree at that moment looks like (word-wrap off):
>> USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
>> ...
>> root     25526  0.0  0.0 106364   636 ?        S    12:37   0:00          \_
>> /bin/sh /usr/sbin/crm_report --dest=/var/log/crm_report -f 0000-01-01 00:00:00
>> root     25585  0.0  0.0 106364   636 ?        S    12:37   0:00
>>   \_ bash /var/log/crm_report/collector
>> root     25613  0.0  0.0 106364   152 ?        S    12:37   0:00
>>       \_ bash /var/log/crm_report/collector
>> root     25614  0.0  0.0 106364   692 ?        S    12:37   0:00
>>           \_ bash /var/log/crm_report/collector
>> root     27965  4.9  0.0 100936   452 ?        S    12:38   0:01
>>           |   \_ cat /var/log/lastlog
>> root     27966 23.0 82.9 3248996 1594688 ?     D    12:38   0:08
>>           |   \_ grep -l -e Starting Pacemaker
>> root     25615  0.0  0.0 155432   600 ?        S    12:37   0:00
>>           \_ sort -u
>>
>> ls -ls /var/log/lastlog shows:
>> 40 -rw-r--r--. 1 root root 23360000876 Sep  8 04:36 /var/log/lastlog
>>
>> That is sparse binary file, which consumes only 40k of disk space.
>> At the same time its size is 23GB, and grep takes all the RAM trying to
>> grep a string from a 23GB of mostly zeroes without new-lines.
>>
>> I believe this is worth fixing,
>
> I guess the UID value is used as offset in the lastlog file (which

exactly
I just should add, that user should be logged-in at least once.

> isOK). When reading such a sparse file, the filesystem should simply
> deliver zero blocks to grep. As grep is designed to read from streads
> there is not much you can do against reading all these zeros, I guess.

yep, I think that another indicator should be used

> Also an mmap based solution might exceed the virtual address space,
> especially for 32-bit systems.
> BTW: Did you try "last Pacemaker"? I could only test with "last
> reboot" here...

That is post-1.1.13

Thanks,
Vladislav