[ClusterLabs] The proxy server received an invalid response from an upstream server.

[Klaus Wenninger]

On 3/16/21 3:18 PM, Ken Gaillot wrote:
> On Tue, 2021-03-16 at 09:42 +0000, Jason Long wrote:
>> Hello,
>> I want to launch a Clustering for my Apache Web Server. I have three
>> servers:
>>
>> 1- Main server that acts as a Reverse Proxy
>> 2- The secondary server that when my main server stopped, work as a
>> Reverse Proxy
>> 3- Apache Web Server
>>
>> The client ---> Reverse Proxy Server ---> Apache Web Server
>>
>> The IP addresses are:
>> Main Server (node1)                    : 192.168.1.3
>> Secondary Server (node2)           : 192.168.1.10
>> Apache Web Server (node3)        : 192.168.1.4
>>
>> On the main and secondary servers, I installed and configured Apache
>> as a Reverse Proxy Server. I created a Virtual Host and my Reverse
>> Configuration is:
>>
>> <VirtualHost *:80>
>>      ProxyPreserveHost On
>>      ProxyPass / http://192.168.1.4/
>>      ProxyPassReverse / http://192.168.1.4/
>> </VirtualHost>
>>
>> As you see, it forward all requests to the Apache Web Server.
>>
>> I installed and configured Corosync and Pacemaker as below:
>>
>> On the main and secondary servers, I opened "/etc/hosts" files and
>> added my servers IP addresses and host names:
>>
>> 192.168.1.3    node1
>> 192.168.1.10  node2
>>
>> Then installed Pacemaker, Corosync, and Pcs packages on both servers
>> and started its service:
>>
>> $ sudo yum install corosync pacemaker pcs
>> $ sudo systemctl enable pcsd
>> $ sudo systemctl start pcsd
>> $ sudo systemctl status pcsd
>>
>> Then Configured the firewall on both servers as below:
>>
>> $ sudo firewall-cmd --permanent --add-service=http
>> $ sudo firewall-cmd --permanent --add-service=high-
>> availability
>> $ sudo firewall-cmd --reload
>>
>> After it, on both servers, I created a password for the "hacluster"
>> user, then on the main server:
>>
>> $ sudo pcs host auth node1 node2 -u hacluster -p password
>> node1: Authorized
>> node2: Authorized
>>
>> Then:
>> $ sudo pcs cluster setup mycluster node1 node2 --start --enable
>> $ sudo pcs cluster enable --all
>> node1: Cluster Enabled
>> node2: Cluster Enabled
>>
>> After it:
>> $ sudo pcs cluster start --all
>> node1: Starting Cluster...
>> node2: Starting Cluster...
>>
>> I checked my clusters with below command and they are up and running:
>> $ sudo pcs status
>> ...
>> Node List:
>>    * Online: [ node1 node2 ]
>> ....
>>
>> And finally, I tried to add a resource:
>> $ sudo pcs resource create floating_ip ocf:heartbeat:IPaddr2
>> ip=192.168.1.4 cidr_netmask=24 op monitor interval=5s
Shouldn't the virtual-IP moved between node1 & node2 be
different from the IP used for the web-server on node3?
And having just one instance of the reverse-proxy that
should probably be colocated with the virtual-IP - right?

Klaus
>> $ sudo pcs resource create http_server ocf:heartbeat:apache
>> configfile="/etc/httpd/conf.d/VirtualHost.conf" op monitor
>> timeout="5s" interval="5s"
>>
>> On both servers (Main and Secondary), pcsd service is enabled, but
>> when I want to see my Apache Web Server then it show me below error:
>>
>> Proxy Error
>> The proxy server received an invalid response from an upstream
>> server.
>> The proxy server could not handle the request
>> Reason: Error reading from remote server
>>
>> Why? Which parts of my configuration is wrong?
>> The output of "sudo pcs status" command is:
>> https://paste.ubuntu.com/p/V9KvHKwKtC/
>>
>> Thank you.
> The thing to investigate is:
>
> Failed Resource Actions:
>    * http_server_start_0 on node2 'error' (1): call=12, status='Timed Out', exitreason='', last-rc-change='2021-03-16 12:28:14 +03:30', queued=0ms, exec=40004ms
>    * http_server_start_0 on node1 'error' (1): call=14, status='Timed Out', exitreason='', last-rc-change='2021-03-16 12:28:52 +03:30', queued=0ms, exec=40002ms
>
> The web server start timed out. Check the system, pacemaker and apache
> logs around those times for any hints.
>
> Did you enable and test the status URL? The ocf:heartbeat:apache agent
> checks the status as part of its monitor (which is also done for
> start). It would be something like:
>
> cat <<-END >/etc/httpd/conf.d/status.conf
>   <Location /server-status>
>      SetHandler server-status
>      Require local
>   </Location>
> END
>