[ClusterLabs] pacemaker remote configuration on ubuntu 14.04

[Ken Gaillot]

On 03/08/2016 11:38 PM, Сергей Филатов wrote:
> ssh -p 3121 compute-1
> ssh_exchange_identification: read: Connection reset by peer
> 
> That’s what I get in /var/log/pacemaker.log after restarting pacemaker_remote:
> Mar 09 05:30:27 [28031] compute-1.domain.com       lrmd:     info: crm_signal_dispatch:  Invoking handler for signal 15: Terminated
> Mar 09 05:30:27 [28031] compute-1.domain.com       lrmd:     info: lrmd_shutdown:        Terminating with  0 clients
> Mar 09 05:30:27 [28031] compute-1.domain.com       lrmd:     info: qb_ipcs_us_withdraw:  withdrawing server sockets
> Mar 09 05:30:27 [28031] compute-1.domain.com       lrmd:     info: crm_xml_cleanup:      Cleaning up memory from libxml2
> Mar 09 05:30:27 [28193] compute-1.domain.com       lrmd:     info: crm_log_init:         Changed active directory to /var/lib/heartbeat/cores/root
> Mar 09 05:30:27 [28193] compute-1.domain.com       lrmd:     info: qb_ipcs_us_publish:   server name: lrmd
> Mar 09 05:30:27 [28193] compute-1.domain.com       lrmd:   notice: lrmd_init_remote_tls_server:  Starting a tls listener on port 3121.
> Mar 09 05:30:28 [28193] compute-1.domain.com       lrmd:   notice: bind_and_listen:      Listening on address ::
> Mar 09 05:30:28 [28193] compute-1.domain.com       lrmd:     info: qb_ipcs_us_publish:   server name: cib_ro
> Mar 09 05:30:28 [28193] compute-1.domain.com       lrmd:     info: qb_ipcs_us_publish:   server name: cib_rw
> Mar 09 05:30:28 [28193] compute-1.domain.com       lrmd:     info: qb_ipcs_us_publish:   server name: cib_shm
> Mar 09 05:30:28 [28193] compute-1.domain.com       lrmd:     info: qb_ipcs_us_publish:   server name: attrd
> Mar 09 05:30:28 [28193] compute-1.domain.com       lrmd:     info: qb_ipcs_us_publish:   server name: stonith-ng
> Mar 09 05:30:28 [28193] compute-1.domain.com       lrmd:     info: qb_ipcs_us_publish:   server name: crmd
> Mar 09 05:30:28 [28193] compute-1.domain.com       lrmd:     info: main:         Starting

It looks like the cluster is not even trying to connect to the remote
node. pacemaker_remote here is binding only to IPv6, so the cluster will
need to contact it on that address.

What is your ocf:pacemaker:remote resource configuration?

Check your cluster node logs for the start action -- if your resource is
named R, the start action will be R_start_0. There will be two nodes of
interest: the node assigned the remote node resource, and the DC.

> I got only pacemaker-remote resource-agents pcs installed, so no /etc/default/pacemaker file on remote node
> selinux is disabled and I specifically opened firewall on 2224, 3121 and 21064 tcp and 5405 udp
> 
>> On 08 Mar 2016, at 08:51, Ken Gaillot <kgaillot at redhat.com> wrote:
>>
>> On 03/07/2016 09:10 PM, Сергей Филатов wrote:
>>> Thanks for an answer. Turned out the problem was not in ipv6.
>>> Remote node is listening on 3121 port and it’s name is resolving fine.
>>> Got authkey file at /etc/pacemaker on both remote and cluster nodes.
>>> What can I check in addition? Is there any walkthrough for ubuntu?
>>
>> Nothing specific to ubuntu, but there's not much distro-specific to it.
>>
>> If you "ssh -p 3121" to the remote node from a cluster node, what do you
>> get?
>>
>> pacemaker_remote will use the usual log settings for pacemaker (probably
>> /var/log/pacemaker.log, probably configured in /etc/default/pacemaker on
>> ubuntu). You should see "New remote connection" in the remote node's log
>> when the cluster tries to connect, and "LRMD client connection
>> established" if it's successful.
>>
>> As always, check for firewall and SELinux issues.
>>
>>>
>>>> On 07 Mar 2016, at 09:40, Ken Gaillot <kgaillot at redhat.com> wrote:
>>>>
>>>> On 03/06/2016 07:43 PM, Сергей Филатов wrote:
>>>>> Hi,
>>>>> I’m trying to set up pacemaker_remote resource on ubuntu 14.04
>>>>> I followed "remote node walkthrough” guide (http://clusterlabs.org/doc/en-US/Pacemaker/1.1/html-single/Pacemaker_Remote/#idm140473081667280 <http://clusterlabs.org/doc/en-US/Pacemaker/1.1/html-single/Pacemaker_Remote/#idm140473081667280>)
>>>>> After creating ocf:pacemaker:remote resource on cluster node, remote node doesn’t show up as online.
>>>>> I guess I need to configure remote agent to listen on ipv4, where can I configure it?
>>>>> Or is there any other steps to set up remote node besides the ones mentioned in guide?
>>>>> tcp6       0      0 :::3121                 :::*                    LISTEN      21620/pacemaker_rem off (0.00/0/0)
>>>>>
>>>>> pacemaker and pacemaker_remote are 1.12 version
>>>>
>>>>
>>>> pacemaker_remote will try to bind to IPv6 addresses first, and only if
>>>> that fails, will it bind to IPv4. There is no way to configure this
>>>> behavior currently, though it obviously would be nice to have.
>>>>
>>>> The only workarounds I can think of are to make IPv6 connections work
>>>> between the cluster and the remote node, or disable IPv6 on the remote
>>>> node. Using IPv6, there could be an issue if your name resolution
>>>> returns both IPv4 and IPv6 addresses for the remote host; you could
>>>> potentially work around that by adding an IPv6-only name for it, and
>>>> using that as the server option to the remote resource.
> 
>