[Pacemaker] [Openais] Linux HA on debian sparc

Steven Dake sdake at redhat.com
Wed Jun 1 21:59:02 UTC 2011


On 06/01/2011 07:42 AM, william felipe_welter wrote:
> Steven,
> 
> cat /proc/meminfo
> ...
> HugePages_Total:       0
> HugePages_Free:        0
> HugePages_Rsvd:        0
> HugePages_Surp:        0
> Hugepagesize:       4096 kB
> ...
> 

It definitely requires a kernel compile and setting the config option to
off.  I don't know the debian way of doing this.

The only reason you may need this option is if you have very large
memory sizes, such as 48GB or more.

Regards
-steve

> Its 4MB..
> 
> How can i disable hugetlb ? ( passing CONFIG_HUGETLBFS=n at boot to
> kernel ?)
> 
> 2011/6/1 Steven Dake <sdake at redhat.com <mailto:sdake at redhat.com>>
> 
>     On 06/01/2011 01:05 AM, Steven Dake wrote:
>     > On 05/31/2011 09:44 PM, Angus Salkeld wrote:
>     >> On Tue, May 31, 2011 at 11:52:48PM -0300, william felipe_welter
>     wrote:
>     >>> Angus,
>     >>>
>     >>> I make some test program (based on the code coreipcc.c) and i
>     now i sure
>     >>> that are problems with the mmap systems call on sparc..
>     >>>
>     >>> Source code of my test program:
>     >>>
>     >>> #include <stdlib.h>
>     >>> #include <sys/mman.h>
>     >>> #include <stdio.h>
>     >>>
>     >>> #define PATH_MAX  36
>     >>>
>     >>> int main()
>     >>> {
>     >>>
>     >>> int32_t fd;
>     >>> void *addr_orig;
>     >>> void *addr;
>     >>> char path[PATH_MAX];
>     >>> const char *file = "teste123XXXXXX";
>     >>> size_t bytes=10024;
>     >>>
>     >>> snprintf (path, PATH_MAX, "/dev/shm/%s", file);
>     >>> printf("PATH=%s\n",path);
>     >>>
>     >>> fd = mkstemp (path);
>     >>> printf("fd=%d \n",fd);
>     >>>
>     >>>
>     >>> addr_orig = mmap (NULL, bytes, PROT_NONE,
>     >>>               MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
>     >>>
>     >>>
>     >>> addr = mmap (addr_orig, bytes, PROT_READ | PROT_WRITE,
>     >>>               MAP_FIXED | MAP_SHARED, fd, 0);
>     >>>
>     >>> printf("ADDR_ORIG:%p  ADDR:%p\n",addr_orig,addr);
>     >>>
>     >>>
>     >>>   if (addr != addr_orig) {
>     >>>                printf("Erro");
>     >>>         }
>     >>> }
>     >>>
>     >>> Results on x86:
>     >>> PATH=/dev/shm/teste123XXXXXX
>     >>> fd=3
>     >>> ADDR_ORIG:0x7f867d8e6000  ADDR:0x7f867d8e6000
>     >>>
>     >>> Results on sparc:
>     >>> PATH=/dev/shm/teste123XXXXXX
>     >>> fd=3
>     >>> ADDR_ORIG:0xf7f72000  ADDR:0xffffffff
>     >>
>     >> Note: 0xffffffff == MAP_FAILED
>     >>
>     >> (from man mmap)
>     >> RETURN VALUE
>     >>        On success, mmap() returns a pointer to the mapped area.  On
>     >>        error, the value MAP_FAILED (that is, (void *) -1) is
>     returned,
>     >>        and errno is  set appropriately.
>     >>
>     >>>
>     >>>
>     >>> But im wondering if is really needed to call mmap 2 times ?
>      What are the
>     >>> reason to call the mmap 2 times, on the second time using the
>     address of the
>     >>> first?
>     >>>
>     >>>
>     >> Well there are 3 calls to mmap()
>     >> 1) one to allocate 2 * what you need (in pages)
>     >> 2) maps the first half of the mem to a real file
>     >> 3) maps the second half of the mem to the same file
>     >>
>     >> The point is when you write to an address over the end of the
>     >> first half of memory it is taken care of the the third mmap which
>     maps
>     >> the address back to the top of the file for you. This means you
>     >> don't have to worry about ringbuffer wrapping which can be a
>     headache.
>     >>
>     >> -Angus
>     >>
>     >
>     > interesting this mmap operation doesn't work on sparc linux.
>     >
>     > Not sure how I can help here - Next step would be a follow up with the
>     > sparc linux mailing list.  I'll do that and cc you on the message
>     - see
>     > if we get any response.
>     >
>     > http://vger.kernel.org/vger-lists.html
>     >
>     >>>
>     >>>
>     >>>
>     >>>
>     >>> 2011/5/31 Angus Salkeld <asalkeld at redhat.com
>     <mailto:asalkeld at redhat.com>>
>     >>>
>     >>>> On Tue, May 31, 2011 at 06:25:56PM -0300, william felipe_welter
>     wrote:
>     >>>>> Thanks Steven,
>     >>>>>
>     >>>>> Now im try to run on the MCP:
>     >>>>> - Uninstall the pacemaker 1.0
>     >>>>> - Compile and install 1.1
>     >>>>>
>     >>>>> But now i have problems to initialize the pacemakerd: Could not
>     >>>> initialize
>     >>>>> Cluster Configuration Database API instance error 2
>     >>>>> Debbuging with gdb i see that the error are on the confdb.. most
>     >>>> specificaly
>     >>>>> the errors start on coreipcc.c  at line:
>     >>>>>
>     >>>>>
>     >>>>> 448        if (addr != addr_orig) {
>     >>>>> 449                goto error_close_unlink;  <- enter here
>     >>>>> 450       }
>     >>>>>
>     >>>>> Some ideia about  what can cause this  ?
>     >>>>>
>     >>>>
>     >>>> I tried porting a ringbuffer (www.libqb.org
>     <http://www.libqb.org>) to sparc and had the same
>     >>>> failure.
>     >>>> There are 3 mmap() calls and on sparc the third one keeps failing.
>     >>>>
>     >>>> This is a common way of creating a ring buffer, see:
>     >>>>
>     http://en.wikipedia.org/wiki/Circular_buffer#Exemplary_POSIX_Implementation
>     >>>>
>     >>>> I couldn't get it working in the short time I tried. It's probably
>     >>>> worth looking at the clib implementation to see why it's failing
>     >>>> (I didn't get to that).
>     >>>>
>     >>>> -Angus
>     >>>>
> 
>     Note, we sorted this out we believe.  Your kernel has hugetlb enabled,
>     probably with 4MB pages.  This requires corosync to allocate 4MB pages.
> 
>     Can you verify your hugetlb settings?
> 
>     If you can turn this option off, you should have atleast a working
>     corosync.
> 
>     Regards
>     -steve
>     >>>>
>     >>>> _______________________________________________
>     >>>> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>     <mailto:Pacemaker at oss.clusterlabs.org>
>     >>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>     >>>>
>     >>>> Project Home: http://www.clusterlabs.org
>     >>>> Getting started:
>     http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>     >>>> Bugs:
>     >>>>
>     http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
>     >>>>
>     >>>
>     >>>
>     >>>
>     >>> --
>     >>> William Felipe Welter
>     >>> ------------------------------
>     >>> Consultor em Tecnologias Livres
>     >>> william.welter at 4linux.com.br <mailto:william.welter at 4linux.com.br>
>     >>> www.4linux.com.br <http://www.4linux.com.br>
>     >>
>     >>> _______________________________________________
>     >>> Openais mailing list
>     >>> Openais at lists.linux-foundation.org
>     <mailto:Openais at lists.linux-foundation.org>
>     >>> https://lists.linux-foundation.org/mailman/listinfo/openais
>     >>
>     >>
>     >> _______________________________________________
>     >> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>     <mailto:Pacemaker at oss.clusterlabs.org>
>     >> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>     >>
>     >> Project Home: http://www.clusterlabs.org
>     >> Getting started:
>     http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>     >> Bugs:
>     http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
>     >
>     > _______________________________________________
>     > Openais mailing list
>     > Openais at lists.linux-foundation.org
>     <mailto:Openais at lists.linux-foundation.org>
>     > https://lists.linux-foundation.org/mailman/listinfo/openais
> 
> 
>     _______________________________________________
>     Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
>     <mailto:Pacemaker at oss.clusterlabs.org>
>     http://oss.clusterlabs.org/mailman/listinfo/pacemaker
> 
>     Project Home: http://www.clusterlabs.org
>     Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>     Bugs:
>     http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
> 
> 
> 
> 
> -- 
> William Felipe Welter
> ------------------------------
> Consultor em Tecnologias Livres
> william.welter at 4linux.com.br <mailto:william.welter at 4linux.com.br>
> www.4linux.com.br <http://www.4linux.com.br>
> 
> 
> 
> _______________________________________________
> Pacemaker mailing list: Pacemaker at oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
> 
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker





More information about the Pacemaker mailing list