[Pacemaker] [Openais] Linux HA on debian sparc

Wed Jun 1 05:31:46 EDT 2011

On 06/01/2011 01:05 AM, Steven Dake wrote:
> On 05/31/2011 09:44 PM, Angus Salkeld wrote:
>> On Tue, May 31, 2011 at 11:52:48PM -0300, william felipe_welter wrote:
>>> Angus,
>>>
>>> I make some test program (based on the code coreipcc.c) and i now i sure
>>> that are problems with the mmap systems call on sparc..
>>>
>>> Source code of my test program:
>>>
>>> #include <stdlib.h>
>>> #include <sys/mman.h>
>>> #include <stdio.h>
>>>
>>> #define PATH_MAX  36
>>>
>>> int main()
>>> {
>>>
>>> int32_t fd;
>>> void *addr_orig;
>>> void *addr;
>>> char path[PATH_MAX];
>>> const char *file = "teste123XXXXXX";
>>> size_t bytes=10024;
>>>
>>> snprintf (path, PATH_MAX, "/dev/shm/%s", file);
>>> printf("PATH=%s\n",path);
>>>
>>> fd = mkstemp (path);
>>> printf("fd=%d \n",fd);
>>>
>>>
>>> addr_orig = mmap (NULL, bytes, PROT_NONE,
>>>               MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
>>>
>>>
>>> addr = mmap (addr_orig, bytes, PROT_READ | PROT_WRITE,
>>>               MAP_FIXED | MAP_SHARED, fd, 0);
>>>
>>> printf("ADDR_ORIG:%p  ADDR:%p\n",addr_orig,addr);
>>>
>>>
>>>   if (addr != addr_orig) {
>>>                printf("Erro");
>>>         }
>>> }
>>>
>>> Results on x86:
>>> PATH=/dev/shm/teste123XXXXXX
>>> fd=3
>>> ADDR_ORIG:0x7f867d8e6000  ADDR:0x7f867d8e6000
>>>
>>> Results on sparc:
>>> PATH=/dev/shm/teste123XXXXXX
>>> fd=3
>>> ADDR_ORIG:0xf7f72000  ADDR:0xffffffff
>>
>> Note: 0xffffffff == MAP_FAILED
>>
>> (from man mmap)
>> RETURN VALUE
>>        On success, mmap() returns a pointer to the mapped area.  On
>>        error, the value MAP_FAILED (that is, (void *) -1) is returned,
>>        and errno is  set appropriately.
>>
>>>
>>>
>>> But im wondering if is really needed to call mmap 2 times ?  What are the
>>> reason to call the mmap 2 times, on the second time using the address of the
>>> first?
>>>
>>>
>> Well there are 3 calls to mmap()
>> 1) one to allocate 2 * what you need (in pages)
>> 2) maps the first half of the mem to a real file
>> 3) maps the second half of the mem to the same file
>>
>> The point is when you write to an address over the end of the
>> first half of memory it is taken care of the the third mmap which maps
>> the address back to the top of the file for you. This means you
>> don't have to worry about ringbuffer wrapping which can be a headache.
>>
>> -Angus
>>
> 
> interesting this mmap operation doesn't work on sparc linux.
> 
> Not sure how I can help here - Next step would be a follow up with the
> sparc linux mailing list.  I'll do that and cc you on the message - see
> if we get any response.
> 
> http://vger.kernel.org/vger-lists.html
> 
>>>
>>>
>>>
>>>
>>> 2011/5/31 Angus Salkeld <asalkeld at redhat.com>
>>>
>>>> On Tue, May 31, 2011 at 06:25:56PM -0300, william felipe_welter wrote:
>>>>> Thanks Steven,
>>>>>
>>>>> Now im try to run on the MCP:
>>>>> - Uninstall the pacemaker 1.0
>>>>> - Compile and install 1.1
>>>>>
>>>>> But now i have problems to initialize the pacemakerd: Could not
>>>> initialize
>>>>> Cluster Configuration Database API instance error 2
>>>>> Debbuging with gdb i see that the error are on the confdb.. most
>>>> specificaly
>>>>> the errors start on coreipcc.c  at line:
>>>>>
>>>>>
>>>>> 448        if (addr != addr_orig) {
>>>>> 449                goto error_close_unlink;  <- enter here
>>>>> 450       }
>>>>>
>>>>> Some ideia about  what can cause this  ?
>>>>>
>>>>
>>>> I tried porting a ringbuffer (www.libqb.org) to sparc and had the same
>>>> failure.
>>>> There are 3 mmap() calls and on sparc the third one keeps failing.
>>>>
>>>> This is a common way of creating a ring buffer, see:
>>>> http://en.wikipedia.org/wiki/Circular_buffer#Exemplary_POSIX_Implementation
>>>>
>>>> I couldn't get it working in the short time I tried. It's probably
>>>> worth looking at the clib implementation to see why it's failing
>>>> (I didn't get to that).
>>>>
>>>> -Angus
>>>>

Note, we sorted this out we believe.  Your kernel has hugetlb enabled,
probably with 4MB pages.  This requires corosync to allocate 4MB pages.

Can you verify your hugetlb settings?

If you can turn this option off, you should have atleast a working corosync.

Regards
-steve
>>>>
>>>> _______________________________________________
>>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>
>>>> Project Home: http://www.clusterlabs.org
>>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>> Bugs:
>>>> http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
>>>>
>>>
>>>
>>>
>>> -- 
>>> William Felipe Welter
>>> ------------------------------
>>> Consultor em Tecnologias Livres
>>> william.welter at 4linux.com.br
>>> www.4linux.com.br
>>
>>> _______________________________________________
>>> Openais mailing list
>>> Openais at lists.linux-foundation.org
>>> https://lists.linux-foundation.org/mailman/listinfo/openais
>>
>>
>> _______________________________________________
>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> Bugs: http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
> 
> _______________________________________________
> Openais mailing list
> Openais at lists.linux-foundation.org
> https://lists.linux-foundation.org/mailman/listinfo/openais