[ClusterLabs] The proxy server received an invalid response from an upstream server.

Jason Long hack3rcon at yahoo.com
Thu Mar 18 10:31:22 EDT 2021 

Thank you, but please forget node3. I want to make a cluster with two nodes. Both nodes has Apache web server and when a node stopped, then another node work.
Can you show me a good tutorial about it? Or can you write it here?






On Thursday, March 18, 2021, 01:40:22 AM GMT+3:30, Ken Gaillot <kgaillot at redhat.com> wrote: 





On Wed, 2021-03-17 at 20:37 +0000, Jason Long wrote:
> The 192.168.1.4 is my secondary VM.
> I want to follow "
> https://www.digitalocean.com/community/tutorials/how-to-set-up-an-apache-active-passive-cluster-using-pacemaker-on-centos-7
> " tutorial.
> At "Step 8 — Adding the Apache Resource", I got below problems:

Verify that you did Step 2 on node3. With the cluster stopped, start
the web server manually, and use curl or wget to verify that you can
successfully get the /server-status URL from the local host.

Step 6 is a bad idea. You should configure and test fencing instead,
before adding resources.

> 1- In the tutorial example, you will see:
> 
> 
> Full list of resources:
>    * Cluster_VIP    (ocf::heartbeat:IPaddr2):  Started webnode01
>    * WebServer  (ocf::heartbeat:apache):    Started webnode02
> 
> 
> But mine:
> 
> 
> Full List of Resources:
>  * Cluster_VIP    (ocf::heartbeat:IPaddr2):    Started node1
>  * WebServer    (ocf::heartbeat:apache):    Starting node2
> 
> 
> Please be careful about "Started" and "Starting" words.
> 
> 
> 2- I wanted to restart the Apache resource, but:
> 
> 
> [root at node1 log]# sudo pcs resource restart WebServer
> Error: crm_resource: Error performing operation: Timer expired
> Set 'WebServer' option: id=WebServer-meta_attributes-target-role
> set=WebServer-meta_attributes name=target-role value=stopped
> Waiting for 1 resources to stop:
>  * WebServer
> Deleted 'WebServer' option: id=WebServer-meta_attributes-target-role
> name=target-role
> Waiting for 1 resources to start again:
>  * WebServer
> Could not complete restart of WebServer, 1 resources remaining
>  * WebServer
> 
> 
> 
> Logs are:
> https://paste.ubuntu.com/p/nHfTRFh4RD/
> 
> 
> 
> Why?
> 
> 
> 
> 
> 
> On Wednesday, March 17, 2021, 11:42:11 PM GMT+3:30, Jason Long <
> hack3rcon at yahoo.com> wrote: 
> 
> 
> 
> 
> 
> Yes, I want Apache always on node3, and then the reverse proxy can
> move between node1 and node2.
> Please see my new efforts.
> 
> 
> 
> 
> 
> 
> On Wednesday, March 17, 2021, 11:38:01 PM GMT+3:30, Ken Gaillot <
> kgaillot at redhat.com> wrote: 
> 
> 
> 
> 
> 
> It sounds to me like your intent is to have apache always on node3,
> and
> then the reverse proxy can move between node1 and node2. The floating
> IP address, which is what users use to contact the site, is
> associated
> with the reverse proxy.
> 
> In any case, the floating IP should be an *additional* IP that is not
> the primary IP address of any host. The cluster will associate this
> IP
> with whichever node is running the proxy.
> 
> If the above scenario is what you want, then you can ban the web
> server
> from node1 and node2, create a group consisting of the floating IP
> and
> the reverse proxy, and ban the group from node3.
> 
> On Wed, 2021-03-17 at 18:51 +0000, Jason Long wrote:
> > Hello,
> > I changed "IP" to my Apache web server:
> > 
> > $ sudo pcs resource update floating_ip ocf:heartbeat:IPaddr2
> > ip=192.168.1.4 cidr_netmask=24 op monitor interval=5s
> > 
> > And did:
> > 
> > $ sudo pcs status
> > Cluster name: mycluster
> > Cluster Summary:
> >  * Stack: corosync
> >  * Current DC: node1 (version 2.0.5-10.fc33-ba59be7122) -
> > partition
> > with quorum
> >  * Last updated: Wed Mar 17 21:55:58 2021
> >  * Last change:  Wed Mar 17 21:55:02 2021 by root via cibadmin on
> > node1
> >  * 2 nodes configured
> >  * 2 resource instances configured
> > 
> > Node List:
> >  * Online: [ node1 node2 ]
> > Full List of Resources:
> >  * floating_ip    (ocf::heartbeat:IPaddr2):    Started node1
> >  * http_server    (ocf::heartbeat:apache):    Stopped
> > 
> > Failed Resource Actions:
> >  * http_server_start_0 on node1 'error' (1): call=10,
> > status='Timed
> > Out', exitreason='', last-rc-change='2021-03-17 21:50:31 +03:30',
> > queued=0ms, exec=40002ms
> >  * http_server_start_0 on node2 'error' (1): call=11,
> > status='Timed
> > Out', exitreason='', last-rc-change='2021-03-17 21:51:11 +03:30',
> > queued=0ms, exec=40002ms
> > 
> > Daemon Status:
> >  corosync: active/enabled
> >  pacemaker: active/enabled
> >  pcsd: active/enabled
> > 
> > 
> > Why "http_server    (ocf::heartbeat:apache):    Stopped" ?
> > 
> > I think you misunderstand my goal, please examine "
> > https://paste.ubuntu.com/p/Nx2ptqZjFg/". I just have one Apache
> > server and two Reverse Proxy servers, when a Reverse Proxy server
> > stopped then another one work.
> > In this scenario, is group resources mandatory?
> > 
> > 
> > 
> > 
> > 
> > On Wednesday, March 17, 2021, 01:50:35 AM GMT+3:30, Reid Wahl <
> > nwahl at redhat.com> wrote: 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > On Tue, Mar 16, 2021 at 3:13 PM Jason Long <hack3rcon at yahoo.com>
> > wrote:
> > > I'm using CentOS.
> > 
> > Ah okay. I think I had made an assumption based on the pastebins
> > URLs.
> > 
> > >  Thus, I must use my Apache web server IP instead of node2?
> > 
> > Yes, it's never a good idea to configure a node's constant IP
> > address
> > within an IPaddr2 resource. That will almost inevitably result in
> > Pacemaker taking down the IP address at some point.
> > 
> > For an IPaddr2 resource, you configure the IP address that's free
> > to
> > move around the cluster. In this case, that's the Apache web server
> > IP. Node2's IP address isn't free to move to node1.
> >  
> > >  About resource group, is you mean "
> > > https://clusterlabs.org/pacemaker/doc/en-US/Pacemaker/2.0/html-single/Pacemaker_Explained/index.html#group-resources
> > > " ?
> > 
> > Yes, that's correct. And if you have access to the Red Hat docs,
> > you
> > can also refer to the following:
> >  - Chapter 5. Configuring an active/passive Apache HTTP server in
> > a
> > Red Hat High Availability cluster (
> > https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/configuring_and_managing_high_availability_clusters/assembly_configuring-active-passive-http-server-in-a-cluster-configuring-and-managing-high-availability-clusters
> > )
> >  - Configuring resource groups (
> > https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/configuring_and_managing_high_availability_clusters/assembly_configuring-cluster-resources-configuring-and-managing-high-availability-clusters#assembly_resource-groups-configuring-cluster-resources
> > )
> > 
> > >  
> > > 
> > > 
> > > 
> > > 
> > > 
> > > On Wednesday, March 17, 2021, 01:10:33 AM GMT+3:30, Reid Wahl <
> > > nwahl at redhat.com> wrote: 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > On Tue, Mar 16, 2021 at 1:47 PM Jason Long <hack3rcon at yahoo.com>
> > > wrote:
> > > > Thanks.
> > > > Excuse me, did you read how did I set my cluster up? Please
> > > > look
> > > > at: https://paste.ubuntu.com/p/Nx2ptqZjFg/
> > > > Which part of my configuration is wrong?
> > > 
> > > 1. You configured the IPaddr2 resource to use node2's IP address
> > > (192.168.1.10) instead of the floating IP address (192.168.1.4).
> > > 2. You didn't configure the resources into a resource group. As a
> > > result, the floating IP may end up on a different node compared
> > > to
> > > the web server.
> > > 
> > > Both of these are explained in more detail in previous emails :)
> > > 
> > > I also thought that Ubuntu used /etc/apache2 instead of
> > > /etc/httpd,
> > > but maybe not.
> > > 
> > > >  Both of the main and secondary servers are an Apache Reverse
> > > > Proxy Server. I want when main server failed, then the
> > > > secondary
> > > > server handle the requests.
> > > >  How can I achieve this goal?
> > > 
> > > I don't know anything about reverse proxies, sorry. I can only
> > > really comment on general principles here, like "an IPaddr2
> > > resource shouldn't manage an IP address that's expected to stay
> > > on
> > > one particular node" and "if two resources need to run on the
> > > same
> > > node and start in a particular order, they need to be grouped."
> > > 
> > > >  
> > > > 
> > > > 
> > > > 
> > > > On Tuesday, March 16, 2021, 11:57:13 PM GMT+3:30, Reid Wahl <
> > > > nwahl at redhat.com> wrote: 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > On Tue, Mar 16, 2021 at 1:03 PM Jason Long <hack3rcon at yahoo.com
> > > > >
> > > > wrote:
> > > > > Thanks.
> > > > > I changed it to the IP address of node2 and I can see my
> > > > > Apache
> > > > > Web Server.
> > > > 
> > > > Like I said, you don't want to do that. You should change it to
> > > > an IP address that you want the cluster to manage. If you set
> > > > it
> > > > to node2's IP address, Pacemaker will try to remove node2's IP
> > > > address and assign it to node1 if the resource fails over to
> > > > node1. If node2 is using that address for anything else (e.g.,
> > > > corosync communication), then that would be a big problem.
> > > > 
> > > > The managed floating IP address should be an IP address
> > > > dedicated
> > > > to the web server, that can move between cluster nodes as
> > > > needed.
> > > > 
> > > > >  
> > > > > $ sudo pcs resource update floating_ip ocf:heartbeat:IPaddr2
> > > > > ip=192.168.1.10 cidr_netmask=24 op monitor interval=5s
> > > > > 
> > > > > Now, I want to test my cluster and stop node1. On node1 I
> > > > > did:
> > > > > 
> > > > > # pcs cluster stop http_server
> > > > > Error: nodes 'http_server' do not appear to exist in
> > > > > configuration
> > > > > 
> > > > > Why?
> > > > 
> > > > The `pcs cluster stop` command stops pacemaker and corosync
> > > > services on a particular node (the local node if you don't
> > > > specify one). You've specified `http_server`, so the command is
> > > > trying to connect to a node called "http_server" and stop
> > > > services there.
> > > > 
> > > > If you want to stop node1, then run `pcs cluster stop node1`.
> > > > 
> > > > If you want to prevent the http_server resource from running
> > > > anywhere, then run `pcs resource disable http_server`.
> > > > 
> > > > If you want to prevent the http_server resource from running on
> > > > node2, then run `pcs resource ban http_server node2`. If you
> > > > want
> > > > to remove that constraint later and allow it to run on node2
> > > > again, run `pcs resource clear http_server`.
> > > > 
> > > > >  
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > On Tuesday, March 16, 2021, 11:05:48 PM GMT+3:30, Reid Wahl <
> > > > > nwahl at redhat.com> wrote: 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > On Tue, Mar 16, 2021 at 12:11 PM Jason Long <
> > > > > hack3rcon at yahoo.com> wrote:
> > > > > > Thank you so much.
> > > > > > I forgot to ask a question. In below command, what should
> > > > > > be
> > > > > > the ip="IP" value? Is it the IP address of my Apache or
> > > > > > node2?
> > > > > > 
> > > > > > $ sudo pcs resource create floating_ip
> > > > > > ocf:heartbeat:IPaddr2
> > > > > > ip="IP" cidr_netmask=24 op monitor interval=5s
> > > > > 
> > > > > It's the IP address that you want the cluster to manage. That
> > > > > sounds like it would be your web server IP address. You
> > > > > definitely don't want to set the ip option to some IP address
> > > > > that resides statically on one of the nodes. An IP managed by
> > > > > an IPaddr2 resource can be moved around the cluster.
> > > > > 
> > > > > If that's your web server IP address, you'll also want to put
> > > > > it in a resource group with your apache resource. Otherwise,
> > > > > the floating IP may end up on a different node from your web
> > > > > server, which renders the IP address useless.
> > > > > 
> > > > > For resources that already exist, you can use the `pcs
> > > > > resource
> > > > > group add` command. For example: `pcs resource group add
> > > > > apache_group floating_ip http_server`.
> > > > > 
> > > > > For resources that you're newly creating, you can use the `
> > > > > --group` option of `pcs resource create`. For example, `pcs
> > > > > resource create new_IP IPaddr2 <options> --group
> > > > > apache_group`.
> > > > > That adds the new resource to the end of the group.
> > > > > 
> > > > > The pcs help outputs have more details on these options.
> > > > > 
> > > > > If you're new to resource groups, you can check them out
> > > > > here: 
> > > > > https://clusterlabs.org/pacemaker/doc/en-US/Pacemaker/2.0/html-single/Pacemaker_Explained/index.html#group-resources
> > > > >  
> > > > > >  
> > > > > > Logs are:
> > > > > > https://paste.ubuntu.com/p/86YHRX6rdC/
> > > > > > https://paste.ubuntu.com/p/HHVzNvhRM2/
> > > > > > https://paste.ubuntu.com/p/kNxynhfyc2/
> > > > > > 
> > > > > > I have not any "status.conf" file:
> > > > > > 
> > > > > > # cat /etc/httpd/conf.d/status.conf
> > > > > > cat: /etc/httpd/conf.d/status.conf: No such file or
> > > > > > directory
> > > > > > 
> > > > > 
> > > > > If you're using Ubuntu, I believe it's in a different
> > > > > location
> > > > > -- somewhere in /etc/apache2 if memory serves.
> > > > > 
> > > > > >  
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > On Tuesday, March 16, 2021, 07:20:32 PM GMT+3:30, Klaus
> > > > > > Wenninger <kwenning at redhat.com> wrote: 
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > On 3/16/21 3:18 PM, Ken Gaillot wrote:
> > > > > > > On Tue, 2021-03-16 at 09:42 +0000, Jason Long wrote:
> > > > > > > > Hello,
> > > > > > > > I want to launch a Clustering for my Apache Web Server.
> > > > > > > > I
> > > > > > > > have three
> > > > > > > > servers:
> > > > > > > > 
> > > > > > > > 1- Main server that acts as a Reverse Proxy
> > > > > > > > 2- The secondary server that when my main server
> > > > > > > > stopped,
> > > > > > > > work as a
> > > > > > > > Reverse Proxy
> > > > > > > > 3- Apache Web Server
> > > > > > > > 
> > > > > > > > The client ---> Reverse Proxy Server ---> Apache Web
> > > > > > > > Server
> > > > > > > > 
> > > > > > > > The IP addresses are:
> > > > > > > > Main Server (node1)                    : 192.168.1.3
> > > > > > > > Secondary Server (node2)          : 192.168.1.10
> > > > > > > > Apache Web Server (node3)        : 192.168.1.4
> > > > > > > > 
> > > > > > > > On the main and secondary servers, I installed and
> > > > > > > > configured Apache
> > > > > > > > as a Reverse Proxy Server. I created a Virtual Host and
> > > > > > > > my Reverse
> > > > > > > > Configuration is:
> > > > > > > > 
> > > > > > > > <VirtualHost *:80>
> > > > > > > >      ProxyPreserveHost On
> > > > > > > >      ProxyPass / http://192.168.1.4/
> > > > > > > >      ProxyPassReverse / http://192.168.1.4/
> > > > > > > > </VirtualHost>
> > > > > > > > 
> > > > > > > > As you see, it forward all requests to the Apache Web
> > > > > > > > Server.
> > > > > > > > 
> > > > > > > > I installed and configured Corosync and Pacemaker as
> > > > > > > > below:
> > > > > > > > 
> > > > > > > > On the main and secondary servers, I opened
> > > > > > > > "/etc/hosts"
> > > > > > > > files and
> > > > > > > > added my servers IP addresses and host names:
> > > > > > > > 
> > > > > > > > 192.168.1.3    node1
> > > > > > > > 192.168.1.10  node2
> > > > > > > > 
> > > > > > > > Then installed Pacemaker, Corosync, and Pcs packages on
> > > > > > > > both servers
> > > > > > > > and started its service:
> > > > > > > > 
> > > > > > > > $ sudo yum install corosync pacemaker pcs
> > > > > > > > $ sudo systemctl enable pcsd
> > > > > > > > $ sudo systemctl start pcsd
> > > > > > > > $ sudo systemctl status pcsd
> > > > > > > > 
> > > > > > > > Then Configured the firewall on both servers as below:
> > > > > > > > 
> > > > > > > > $ sudo firewall-cmd --permanent --add-service=http
> > > > > > > > $ sudo firewall-cmd --permanent --add-service=high-
> > > > > > > > availability
> > > > > > > > $ sudo firewall-cmd --reload
> > > > > > > > 
> > > > > > > > After it, on both servers, I created a password for the
> > > > > > > > "hacluster"
> > > > > > > > user, then on the main server:
> > > > > > > > 
> > > > > > > > $ sudo pcs host auth node1 node2 -u hacluster -p
> > > > > > > > password
> > > > > > > > node1: Authorized
> > > > > > > > node2: Authorized
> > > > > > > > 
> > > > > > > > Then:
> > > > > > > > $ sudo pcs cluster setup mycluster node1 node2 --start
> > > > > > > > --
> > > > > > > > enable
> > > > > > > > $ sudo pcs cluster enable --all
> > > > > > > > node1: Cluster Enabled
> > > > > > > > node2: Cluster Enabled
> > > > > > > > 
> > > > > > > > After it:
> > > > > > > > $ sudo pcs cluster start --all
> > > > > > > > node1: Starting Cluster...
> > > > > > > > node2: Starting Cluster...
> > > > > > > > 
> > > > > > > > I checked my clusters with below command and they are
> > > > > > > > up
> > > > > > > > and running:
> > > > > > > > $ sudo pcs status
> > > > > > > > ...
> > > > > > > > Node List:
> > > > > > > >    * Online: [ node1 node2 ]
> > > > > > > > ....
> > > > > > > > 
> > > > > > > > And finally, I tried to add a resource:
> > > > > > > > $ sudo pcs resource create floating_ip
> > > > > > > > ocf:heartbeat:IPaddr2
> > > > > > > > ip=192.168.1.4 cidr_netmask=24 op monitor interval=5s
> > > > > > 
> > > > > > Shouldn't the virtual-IP moved between node1 & node2 be
> > > > > > different from the IP used for the web-server on node3?
> > > > > > And having just one instance of the reverse-proxy that
> > > > > > should probably be colocated with the virtual-IP - right?
> > > > > > 
> > > > > > Klaus
> > > > > > 
> > > > > > > > $ sudo pcs resource create http_server
> > > > > > > > ocf:heartbeat:apache
> > > > > > > > configfile="/etc/httpd/conf.d/VirtualHost.conf" op
> > > > > > > > monitor
> > > > > > > > timeout="5s" interval="5s"
> > > > > > > > 
> > > > > > > > On both servers (Main and Secondary), pcsd service is
> > > > > > > > enabled, but
> > > > > > > > when I want to see my Apache Web Server then it show me
> > > > > > > > below error:
> > > > > > > > 
> > > > > > > > Proxy Error
> > > > > > > > The proxy server received an invalid response from an
> > > > > > > > upstream
> > > > > > > > server.
> > > > > > > > The proxy server could not handle the request
> > > > > > > > Reason: Error reading from remote server
> > > > > > > > 
> > > > > > > > Why? Which parts of my configuration is wrong?
> > > > > > > > The output of "sudo pcs status" command is:
> > > > > > > > https://paste.ubuntu.com/p/V9KvHKwKtC/
> > > > > > > > 
> > > > > > > > Thank you.
> > > > > > > 
> > > > > > > The thing to investigate is:
> > > > > > > 
> > > > > > > Failed Resource Actions:
> > > > > > >    * http_server_start_0 on node2 'error' (1): call=12,
> > > > > > > status='Timed Out', exitreason='', last-rc-change='2021-
> > > > > > > 03-
> > > > > > > 16 12:28:14 +03:30', queued=0ms, exec=40004ms
> > > > > > >    * http_server_start_0 on node1 'error' (1): call=14,
> > > > > > > status='Timed Out', exitreason='', last-rc-change='2021-
> > > > > > > 03-
> > > > > > > 16 12:28:52 +03:30', queued=0ms, exec=40002ms
> > > > > > > 
> > > > > > > The web server start timed out. Check the system,
> > > > > > > pacemaker
> > > > > > > and apache
> > > > > > > logs around those times for any hints.
> > > > > > > 
> > > > > > > Did you enable and test the status URL? The
> > > > > > > ocf:heartbeat:apache agent
> > > > > > > checks the status as part of its monitor (which is also
> > > > > > > done for
> > > > > > > start). It would be something like:
> > > > > > > 
> > > > > > > cat <<-END >/etc/httpd/conf.d/status.conf
> > > > > > >  <Location /server-status>
> > > > > > >      SetHandler server-status
> > > > > > >      Require local
> > > > > > >  </Location>
> > > > > > > END
> > > > > > > 
> > > > > > 
> > > > > > _______________________________________________
> > > > > > Manage your subscription:
> > > > > > https://lists.clusterlabs.org/mailman/listinfo/users
> > > > > > 
> > > > > > ClusterLabs home: https://www.clusterlabs.org/
> > > > > > 
> > > > > > _______________________________________________
> > > > > > Manage your subscription:
> > > > > > https://lists.clusterlabs.org/mailman/listinfo/users
> > > > > > 
> > > > > > ClusterLabs home: https://www.clusterlabs.org/
> > > > > > 
> > > > > > 
> > > > > 
> > > > > 
> > > > > -- 
> > > > > Regards,
> > > > > 
> > > > > Reid Wahl, RHCA
> > > > > Senior Software Maintenance Engineer, Red Hat
> > > > > CEE - Platform Support Delivery - ClusterHA
> > > > > 
> > > > > 
> > > > > _______________________________________________
> > > > > Manage your subscription:
> > > > > https://lists.clusterlabs.org/mailman/listinfo/users
> > > > > 
> > > > > ClusterLabs home: https://www.clusterlabs.org/
> > > > > _______________________________________________
> > > > > Manage your subscription:
> > > > > https://lists.clusterlabs.org/mailman/listinfo/users
> > > > > 
> > > > > ClusterLabs home: https://www.clusterlabs.org/
> > > > > 
> > > > 
> > > > 
> > > > -- 
> > > > Regards,
> > > > 
> > > > Reid Wahl, RHCA
> > > > Senior Software Maintenance Engineer, Red Hat
> > > > CEE - Platform Support Delivery - ClusterHA
> > > > 
> > > > _______________________________________________
> > > > Manage your subscription:
> > > > https://lists.clusterlabs.org/mailman/listinfo/users
> > > > 
> > > > ClusterLabs home: https://www.clusterlabs.org/
> > > > _______________________________________________
> > > > Manage your subscription:
> > > > https://lists.clusterlabs.org/mailman/listinfo/users
> > > > 
> > > > ClusterLabs home: https://www.clusterlabs.org/
> > > > 
> > > 
> > > 
> > > -- 
> > > Regards,
> > > 
> > > Reid Wahl, RHCA
> > > Senior Software Maintenance Engineer, Red Hat
> > > CEE - Platform Support Delivery - ClusterHA
> > > 
> > > _______________________________________________
> > > Manage your subscription:
> > > https://lists.clusterlabs.org/mailman/listinfo/users
> > > 
> > > ClusterLabs home: https://www.clusterlabs.org/
> > > _______________________________________________
> > > Manage your subscription:
> > > https://lists.clusterlabs.org/mailman/listinfo/users
> > > 
> > > ClusterLabs home: https://www.clusterlabs.org/
> > > 
> > 
> > 
-- 
Ken Gaillot <kgaillot at redhat.com>

_______________________________________________
Manage your subscription:
https://lists.clusterlabs.org/mailman/listinfo/users

ClusterLabs home: https://www.clusterlabs.org/

More information about the Users mailing list