[Pacemaker] [Openais] Linux HA on debian sparc
william felipe_welter
wfelipew at gmail.com
Thu Jun 2 02:05:35 EDT 2011
I recompile my kernel without hugetlb .. and the result are the same..
My test program still resulting:
PATH=/dev/shm/teste123XXXXXX
page size=20000
fd=3
ADDR_ORIG:0xe000a000 ADDR:0xffffffff
Erro
And Pacemaker still resulting because the mmap error:
Could not initialize Cluster Configuration Database API instance error 2
For make sure that i have disable the hugetlb there is my /proc/meminfo:
MemTotal: 33093488 kB
MemFree: 32855616 kB
Buffers: 5600 kB
Cached: 53480 kB
SwapCached: 0 kB
Active: 45768 kB
Inactive: 28104 kB
Active(anon): 18024 kB
Inactive(anon): 1560 kB
Active(file): 27744 kB
Inactive(file): 26544 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 6104680 kB
SwapFree: 6104680 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 14936 kB
Mapped: 7736 kB
Shmem: 4624 kB
Slab: 39184 kB
SReclaimable: 10088 kB
SUnreclaim: 29096 kB
KernelStack: 7088 kB
PageTables: 1160 kB
Quicklists: 17664 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 22651424 kB
Committed_AS: 519368 kB
VmallocTotal: 1069547520 kB
VmallocUsed: 11064 kB
VmallocChunk: 1069529616 kB
2011/6/1 Steven Dake <sdake at redhat.com>:
> On 06/01/2011 07:42 AM, william felipe_welter wrote:
>> Steven,
>>
>> cat /proc/meminfo
>> ...
>> HugePages_Total: 0
>> HugePages_Free: 0
>> HugePages_Rsvd: 0
>> HugePages_Surp: 0
>> Hugepagesize: 4096 kB
>> ...
>>
>
> It definitely requires a kernel compile and setting the config option to
> off. I don't know the debian way of doing this.
>
> The only reason you may need this option is if you have very large
> memory sizes, such as 48GB or more.
>
> Regards
> -steve
>
>> Its 4MB..
>>
>> How can i disable hugetlb ? ( passing CONFIG_HUGETLBFS=n at boot to
>> kernel ?)
>>
>> 2011/6/1 Steven Dake <sdake at redhat.com <mailto:sdake at redhat.com>>
>>
>> On 06/01/2011 01:05 AM, Steven Dake wrote:
>> > On 05/31/2011 09:44 PM, Angus Salkeld wrote:
>> >> On Tue, May 31, 2011 at 11:52:48PM -0300, william felipe_welter
>> wrote:
>> >>> Angus,
>> >>>
>> >>> I make some test program (based on the code coreipcc.c) and i
>> now i sure
>> >>> that are problems with the mmap systems call on sparc..
>> >>>
>> >>> Source code of my test program:
>> >>>
>> >>> #include <stdlib.h>
>> >>> #include <sys/mman.h>
>> >>> #include <stdio.h>
>> >>>
>> >>> #define PATH_MAX 36
>> >>>
>> >>> int main()
>> >>> {
>> >>>
>> >>> int32_t fd;
>> >>> void *addr_orig;
>> >>> void *addr;
>> >>> char path[PATH_MAX];
>> >>> const char *file = "teste123XXXXXX";
>> >>> size_t bytes=10024;
>> >>>
>> >>> snprintf (path, PATH_MAX, "/dev/shm/%s", file);
>> >>> printf("PATH=%s\n",path);
>> >>>
>> >>> fd = mkstemp (path);
>> >>> printf("fd=%d \n",fd);
>> >>>
>> >>>
>> >>> addr_orig = mmap (NULL, bytes, PROT_NONE,
>> >>> MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
>> >>>
>> >>>
>> >>> addr = mmap (addr_orig, bytes, PROT_READ | PROT_WRITE,
>> >>> MAP_FIXED | MAP_SHARED, fd, 0);
>> >>>
>> >>> printf("ADDR_ORIG:%p ADDR:%p\n",addr_orig,addr);
>> >>>
>> >>>
>> >>> if (addr != addr_orig) {
>> >>> printf("Erro");
>> >>> }
>> >>> }
>> >>>
>> >>> Results on x86:
>> >>> PATH=/dev/shm/teste123XXXXXX
>> >>> fd=3
>> >>> ADDR_ORIG:0x7f867d8e6000 ADDR:0x7f867d8e6000
>> >>>
>> >>> Results on sparc:
>> >>> PATH=/dev/shm/teste123XXXXXX
>> >>> fd=3
>> >>> ADDR_ORIG:0xf7f72000 ADDR:0xffffffff
>> >>
>> >> Note: 0xffffffff == MAP_FAILED
>> >>
>> >> (from man mmap)
>> >> RETURN VALUE
>> >> On success, mmap() returns a pointer to the mapped area. On
>> >> error, the value MAP_FAILED (that is, (void *) -1) is
>> returned,
>> >> and errno is set appropriately.
>> >>
>> >>>
>> >>>
>> >>> But im wondering if is really needed to call mmap 2 times ?
>> What are the
>> >>> reason to call the mmap 2 times, on the second time using the
>> address of the
>> >>> first?
>> >>>
>> >>>
>> >> Well there are 3 calls to mmap()
>> >> 1) one to allocate 2 * what you need (in pages)
>> >> 2) maps the first half of the mem to a real file
>> >> 3) maps the second half of the mem to the same file
>> >>
>> >> The point is when you write to an address over the end of the
>> >> first half of memory it is taken care of the the third mmap which
>> maps
>> >> the address back to the top of the file for you. This means you
>> >> don't have to worry about ringbuffer wrapping which can be a
>> headache.
>> >>
>> >> -Angus
>> >>
>> >
>> > interesting this mmap operation doesn't work on sparc linux.
>> >
>> > Not sure how I can help here - Next step would be a follow up with the
>> > sparc linux mailing list. I'll do that and cc you on the message
>> - see
>> > if we get any response.
>> >
>> > http://vger.kernel.org/vger-lists.html
>> >
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> 2011/5/31 Angus Salkeld <asalkeld at redhat.com
>> <mailto:asalkeld at redhat.com>>
>> >>>
>> >>>> On Tue, May 31, 2011 at 06:25:56PM -0300, william felipe_welter
>> wrote:
>> >>>>> Thanks Steven,
>> >>>>>
>> >>>>> Now im try to run on the MCP:
>> >>>>> - Uninstall the pacemaker 1.0
>> >>>>> - Compile and install 1.1
>> >>>>>
>> >>>>> But now i have problems to initialize the pacemakerd: Could not
>> >>>> initialize
>> >>>>> Cluster Configuration Database API instance error 2
>> >>>>> Debbuging with gdb i see that the error are on the confdb.. most
>> >>>> specificaly
>> >>>>> the errors start on coreipcc.c at line:
>> >>>>>
>> >>>>>
>> >>>>> 448 if (addr != addr_orig) {
>> >>>>> 449 goto error_close_unlink; <- enter here
>> >>>>> 450 }
>> >>>>>
>> >>>>> Some ideia about what can cause this ?
>> >>>>>
>> >>>>
>> >>>> I tried porting a ringbuffer (www.libqb.org
>> <http://www.libqb.org>) to sparc and had the same
>> >>>> failure.
>> >>>> There are 3 mmap() calls and on sparc the third one keeps failing.
>> >>>>
>> >>>> This is a common way of creating a ring buffer, see:
>> >>>>
>> http://en.wikipedia.org/wiki/Circular_buffer#Exemplary_POSIX_Implementation
>> >>>>
>> >>>> I couldn't get it working in the short time I tried. It's probably
>> >>>> worth looking at the clib implementation to see why it's failing
>> >>>> (I didn't get to that).
>> >>>>
>> >>>> -Angus
>> >>>>
>>
>> Note, we sorted this out we believe. Your kernel has hugetlb enabled,
>> probably with 4MB pages. This requires corosync to allocate 4MB pages.
>>
>> Can you verify your hugetlb settings?
>>
>> If you can turn this option off, you should have atleast a working
>> corosync.
>>
>> Regards
>> -steve
>> >>>>
>> >>>> _______________________________________________
>> >>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>> <mailto:Pacemaker at oss.clusterlabs.org>
>> >>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>> >>>>
>> >>>> Project Home: http://www.clusterlabs.org
>> >>>> Getting started:
>> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> >>>> Bugs:
>> >>>>
>> http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> William Felipe Welter
>> >>> ------------------------------
>> >>> Consultor em Tecnologias Livres
>> >>> william.welter at 4linux.com.br <mailto:william.welter at 4linux.com.br>
>> >>> www.4linux.com.br <http://www.4linux.com.br>
>> >>
>> >>> _______________________________________________
>> >>> Openais mailing list
>> >>> Openais at lists.linux-foundation.org
>> <mailto:Openais at lists.linux-foundation.org>
>> >>> https://lists.linux-foundation.org/mailman/listinfo/openais
>> >>
>> >>
>> >> _______________________________________________
>> >> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>> <mailto:Pacemaker at oss.clusterlabs.org>
>> >> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>> >>
>> >> Project Home: http://www.clusterlabs.org
>> >> Getting started:
>> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> >> Bugs:
>> http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
>> >
>> > _______________________________________________
>> > Openais mailing list
>> > Openais at lists.linux-foundation.org
>> <mailto:Openais at lists.linux-foundation.org>
>> > https://lists.linux-foundation.org/mailman/listinfo/openais
>>
>>
>> _______________________________________________
>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>> <mailto:Pacemaker at oss.clusterlabs.org>
>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> Bugs:
>> http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
>>
>>
>>
>>
>> --
>> William Felipe Welter
>> ------------------------------
>> Consultor em Tecnologias Livres
>> william.welter at 4linux.com.br <mailto:william.welter at 4linux.com.br>
>> www.4linux.com.br <http://www.4linux.com.br>
>>
>>
>>
>> _______________________________________________
>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> Bugs: http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
>
>
--
William Felipe Welter
------------------------------
Consultor em Tecnologias Livres
william.welter at 4linux.com.br
www.4linux.com.br
More information about the Pacemaker
mailing list