[ClusterLabs] The proxy server received an invalid response from an upstream server.
Ken Gaillot
kgaillot at redhat.com
Thu Mar 18 10:45:39 EDT 2021
"Clusters from Scratch" is exactly that. The currently published one is
based on CentOS 7:
https://clusterlabs.org/pacemaker/doc/en-US/Pacemaker/2.0/html-single/Clusters_from_Scratch/
A new one based on CentOS Stream 8 will be published with the next
release. A preview is available:
http://people.redhat.com/~kgaillot/pacemaker/doc/2.0/Clusters_from_Scratch/html/
You only need to do through Chapter 7 to get a clustered web server
working. After that, you need some way to synchronize the web
documents; the walk-through demonstrates DRBD and GFS, but you can use
any means available to you.
On Thu, 2021-03-18 at 14:31 +0000, Jason Long wrote:
> Thank you, but please forget node3. I want to make a cluster with two
> nodes. Both nodes has Apache web server and when a node stopped, then
> another node work.
> Can you show me a good tutorial about it? Or can you write it here?
>
>
>
>
>
>
> On Thursday, March 18, 2021, 01:40:22 AM GMT+3:30, Ken Gaillot <
> kgaillot at redhat.com> wrote:
>
>
>
>
>
> On Wed, 2021-03-17 at 20:37 +0000, Jason Long wrote:
> > The 192.168.1.4 is my secondary VM.
> > I want to follow "
> > https://www.digitalocean.com/community/tutorials/how-to-set-up-an-apache-active-passive-cluster-using-pacemaker-on-centos-7
> > " tutorial.
> > At "Step 8 — Adding the Apache Resource", I got below problems:
>
> Verify that you did Step 2 on node3. With the cluster stopped, start
> the web server manually, and use curl or wget to verify that you can
> successfully get the /server-status URL from the local host.
>
> Step 6 is a bad idea. You should configure and test fencing instead,
> before adding resources.
>
> > 1- In the tutorial example, you will see:
> >
> >
> > Full list of resources:
> > * Cluster_VIP (ocf::heartbeat:IPaddr2): Started webnode01
> > * WebServer (ocf::heartbeat:apache): Started webnode02
> >
> >
> > But mine:
> >
> >
> > Full List of Resources:
> > * Cluster_VIP (ocf::heartbeat:IPaddr2): Started node1
> > * WebServer (ocf::heartbeat:apache): Starting node2
> >
> >
> > Please be careful about "Started" and "Starting" words.
> >
> >
> > 2- I wanted to restart the Apache resource, but:
> >
> >
> > [root at node1 log]# sudo pcs resource restart WebServer
> > Error: crm_resource: Error performing operation: Timer expired
> > Set 'WebServer' option: id=WebServer-meta_attributes-target-role
> > set=WebServer-meta_attributes name=target-role value=stopped
> > Waiting for 1 resources to stop:
> > * WebServer
> > Deleted 'WebServer' option: id=WebServer-meta_attributes-target-
> > role
> > name=target-role
> > Waiting for 1 resources to start again:
> > * WebServer
> > Could not complete restart of WebServer, 1 resources remaining
> > * WebServer
> >
> >
> >
> > Logs are:
> > https://paste.ubuntu.com/p/nHfTRFh4RD/
> >
> >
> >
> > Why?
> >
> >
> >
> >
> >
> > On Wednesday, March 17, 2021, 11:42:11 PM GMT+3:30, Jason Long <
> > hack3rcon at yahoo.com> wrote:
> >
> >
> >
> >
> >
> > Yes, I want Apache always on node3, and then the reverse proxy can
> > move between node1 and node2.
> > Please see my new efforts.
> >
> >
> >
> >
> >
> >
> > On Wednesday, March 17, 2021, 11:38:01 PM GMT+3:30, Ken Gaillot <
> > kgaillot at redhat.com> wrote:
> >
> >
> >
> >
> >
> > It sounds to me like your intent is to have apache always on node3,
> > and
> > then the reverse proxy can move between node1 and node2. The
> > floating
> > IP address, which is what users use to contact the site, is
> > associated
> > with the reverse proxy.
> >
> > In any case, the floating IP should be an *additional* IP that is
> > not
> > the primary IP address of any host. The cluster will associate this
> > IP
> > with whichever node is running the proxy.
> >
> > If the above scenario is what you want, then you can ban the web
> > server
> > from node1 and node2, create a group consisting of the floating IP
> > and
> > the reverse proxy, and ban the group from node3.
> >
> > On Wed, 2021-03-17 at 18:51 +0000, Jason Long wrote:
> > > Hello,
> > > I changed "IP" to my Apache web server:
> > >
> > > $ sudo pcs resource update floating_ip ocf:heartbeat:IPaddr2
> > > ip=192.168.1.4 cidr_netmask=24 op monitor interval=5s
> > >
> > > And did:
> > >
> > > $ sudo pcs status
> > > Cluster name: mycluster
> > > Cluster Summary:
> > > * Stack: corosync
> > > * Current DC: node1 (version 2.0.5-10.fc33-ba59be7122) -
> > > partition
> > > with quorum
> > > * Last updated: Wed Mar 17 21:55:58 2021
> > > * Last change: Wed Mar 17 21:55:02 2021 by root via cibadmin
> > > on
> > > node1
> > > * 2 nodes configured
> > > * 2 resource instances configured
> > >
> > > Node List:
> > > * Online: [ node1 node2 ]
> > > Full List of Resources:
> > > * floating_ip (ocf::heartbeat:IPaddr2): Started node1
> > > * http_server (ocf::heartbeat:apache): Stopped
> > >
> > > Failed Resource Actions:
> > > * http_server_start_0 on node1 'error' (1): call=10,
> > > status='Timed
> > > Out', exitreason='', last-rc-change='2021-03-17 21:50:31 +03:30',
> > > queued=0ms, exec=40002ms
> > > * http_server_start_0 on node2 'error' (1): call=11,
> > > status='Timed
> > > Out', exitreason='', last-rc-change='2021-03-17 21:51:11 +03:30',
> > > queued=0ms, exec=40002ms
> > >
> > > Daemon Status:
> > > corosync: active/enabled
> > > pacemaker: active/enabled
> > > pcsd: active/enabled
> > >
> > >
> > > Why "http_server (ocf::heartbeat:apache): Stopped" ?
> > >
> > > I think you misunderstand my goal, please examine "
> > > https://paste.ubuntu.com/p/Nx2ptqZjFg/". I just have one Apache
> > > server and two Reverse Proxy servers, when a Reverse Proxy server
> > > stopped then another one work.
> > > In this scenario, is group resources mandatory?
> > >
> > >
> > >
> > >
> > >
> > > On Wednesday, March 17, 2021, 01:50:35 AM GMT+3:30, Reid Wahl <
> > > nwahl at redhat.com> wrote:
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Tue, Mar 16, 2021 at 3:13 PM Jason Long <hack3rcon at yahoo.com>
> > > wrote:
> > > > I'm using CentOS.
> > >
> > > Ah okay. I think I had made an assumption based on the pastebins
> > > URLs.
> > >
> > > > Thus, I must use my Apache web server IP instead of node2?
> > >
> > > Yes, it's never a good idea to configure a node's constant IP
> > > address
> > > within an IPaddr2 resource. That will almost inevitably result in
> > > Pacemaker taking down the IP address at some point.
> > >
> > > For an IPaddr2 resource, you configure the IP address that's free
> > > to
> > > move around the cluster. In this case, that's the Apache web
> > > server
> > > IP. Node2's IP address isn't free to move to node1.
> > >
> > > > About resource group, is you mean "
> > > > https://clusterlabs.org/pacemaker/doc/en-US/Pacemaker/2.0/html-single/Pacemaker_Explained/index.html#group-resources
> > > > " ?
> > >
> > > Yes, that's correct. And if you have access to the Red Hat docs,
> > > you
> > > can also refer to the following:
> > > - Chapter 5. Configuring an active/passive Apache HTTP server
> > > in
> > > a
> > > Red Hat High Availability cluster (
> > > https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/configuring_and_managing_high_availability_clusters/assembly_configuring-active-passive-http-server-in-a-cluster-configuring-and-managing-high-availability-clusters
> > > )
> > > - Configuring resource groups (
> > > https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/configuring_and_managing_high_availability_clusters/assembly_configuring-cluster-resources-configuring-and-managing-high-availability-clusters#assembly_resource-groups-configuring-cluster-resources
> > > )
> > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Wednesday, March 17, 2021, 01:10:33 AM GMT+3:30, Reid Wahl <
> > > > nwahl at redhat.com> wrote:
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Tue, Mar 16, 2021 at 1:47 PM Jason Long <hack3rcon at yahoo.com
> > > > >
> > > > wrote:
> > > > > Thanks.
> > > > > Excuse me, did you read how did I set my cluster up? Please
> > > > > look
> > > > > at: https://paste.ubuntu.com/p/Nx2ptqZjFg/
> > > > > Which part of my configuration is wrong?
> > > >
> > > > 1. You configured the IPaddr2 resource to use node2's IP
> > > > address
> > > > (192.168.1.10) instead of the floating IP address
> > > > (192.168.1.4).
> > > > 2. You didn't configure the resources into a resource group. As
> > > > a
> > > > result, the floating IP may end up on a different node compared
> > > > to
> > > > the web server.
> > > >
> > > > Both of these are explained in more detail in previous emails
> > > > :)
> > > >
> > > > I also thought that Ubuntu used /etc/apache2 instead of
> > > > /etc/httpd,
> > > > but maybe not.
> > > >
> > > > > Both of the main and secondary servers are an Apache
> > > > > Reverse
> > > > > Proxy Server. I want when main server failed, then the
> > > > > secondary
> > > > > server handle the requests.
> > > > > How can I achieve this goal?
> > > >
> > > > I don't know anything about reverse proxies, sorry. I can only
> > > > really comment on general principles here, like "an IPaddr2
> > > > resource shouldn't manage an IP address that's expected to stay
> > > > on
> > > > one particular node" and "if two resources need to run on the
> > > > same
> > > > node and start in a particular order, they need to be grouped."
> > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Tuesday, March 16, 2021, 11:57:13 PM GMT+3:30, Reid Wahl <
> > > > > nwahl at redhat.com> wrote:
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Tue, Mar 16, 2021 at 1:03 PM Jason Long <
> > > > > hack3rcon at yahoo.com
> > > > > >
> > > > >
> > > > > wrote:
> > > > > > Thanks.
> > > > > > I changed it to the IP address of node2 and I can see my
> > > > > > Apache
> > > > > > Web Server.
> > > > >
> > > > > Like I said, you don't want to do that. You should change it
> > > > > to
> > > > > an IP address that you want the cluster to manage. If you set
> > > > > it
> > > > > to node2's IP address, Pacemaker will try to remove node2's
> > > > > IP
> > > > > address and assign it to node1 if the resource fails over to
> > > > > node1. If node2 is using that address for anything else
> > > > > (e.g.,
> > > > > corosync communication), then that would be a big problem.
> > > > >
> > > > > The managed floating IP address should be an IP address
> > > > > dedicated
> > > > > to the web server, that can move between cluster nodes as
> > > > > needed.
> > > > >
> > > > > >
> > > > > > $ sudo pcs resource update floating_ip
> > > > > > ocf:heartbeat:IPaddr2
> > > > > > ip=192.168.1.10 cidr_netmask=24 op monitor interval=5s
> > > > > >
> > > > > > Now, I want to test my cluster and stop node1. On node1 I
> > > > > > did:
> > > > > >
> > > > > > # pcs cluster stop http_server
> > > > > > Error: nodes 'http_server' do not appear to exist in
> > > > > > configuration
> > > > > >
> > > > > > Why?
> > > > >
> > > > > The `pcs cluster stop` command stops pacemaker and corosync
> > > > > services on a particular node (the local node if you don't
> > > > > specify one). You've specified `http_server`, so the command
> > > > > is
> > > > > trying to connect to a node called "http_server" and stop
> > > > > services there.
> > > > >
> > > > > If you want to stop node1, then run `pcs cluster stop node1`.
> > > > >
> > > > > If you want to prevent the http_server resource from running
> > > > > anywhere, then run `pcs resource disable http_server`.
> > > > >
> > > > > If you want to prevent the http_server resource from running
> > > > > on
> > > > > node2, then run `pcs resource ban http_server node2`. If you
> > > > > want
> > > > > to remove that constraint later and allow it to run on node2
> > > > > again, run `pcs resource clear http_server`.
> > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Tuesday, March 16, 2021, 11:05:48 PM GMT+3:30, Reid Wahl
> > > > > > <
> > > > > > nwahl at redhat.com> wrote:
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Tue, Mar 16, 2021 at 12:11 PM Jason Long <
> > > > > > hack3rcon at yahoo.com> wrote:
> > > > > > > Thank you so much.
> > > > > > > I forgot to ask a question. In below command, what should
> > > > > > > be
> > > > > > > the ip="IP" value? Is it the IP address of my Apache or
> > > > > > > node2?
> > > > > > >
> > > > > > > $ sudo pcs resource create floating_ip
> > > > > > > ocf:heartbeat:IPaddr2
> > > > > > > ip="IP" cidr_netmask=24 op monitor interval=5s
> > > > > >
> > > > > > It's the IP address that you want the cluster to manage.
> > > > > > That
> > > > > > sounds like it would be your web server IP address. You
> > > > > > definitely don't want to set the ip option to some IP
> > > > > > address
> > > > > > that resides statically on one of the nodes. An IP managed
> > > > > > by
> > > > > > an IPaddr2 resource can be moved around the cluster.
> > > > > >
> > > > > > If that's your web server IP address, you'll also want to
> > > > > > put
> > > > > > it in a resource group with your apache resource.
> > > > > > Otherwise,
> > > > > > the floating IP may end up on a different node from your
> > > > > > web
> > > > > > server, which renders the IP address useless.
> > > > > >
> > > > > > For resources that already exist, you can use the `pcs
> > > > > > resource
> > > > > > group add` command. For example: `pcs resource group add
> > > > > > apache_group floating_ip http_server`.
> > > > > >
> > > > > > For resources that you're newly creating, you can use the `
> > > > > > --group` option of `pcs resource create`. For example, `pcs
> > > > > > resource create new_IP IPaddr2 <options> --group
> > > > > > apache_group`.
> > > > > > That adds the new resource to the end of the group.
> > > > > >
> > > > > > The pcs help outputs have more details on these options.
> > > > > >
> > > > > > If you're new to resource groups, you can check them out
> > > > > > here:
> > > > > > https://clusterlabs.org/pacemaker/doc/en-US/Pacemaker/2.0/html-single/Pacemaker_Explained/index.html#group-resources
> > > > > >
> > > > > > >
> > > > > > > Logs are:
> > > > > > > https://paste.ubuntu.com/p/86YHRX6rdC/
> > > > > > > https://paste.ubuntu.com/p/HHVzNvhRM2/
> > > > > > > https://paste.ubuntu.com/p/kNxynhfyc2/
> > > > > > >
> > > > > > > I have not any "status.conf" file:
> > > > > > >
> > > > > > > # cat /etc/httpd/conf.d/status.conf
> > > > > > > cat: /etc/httpd/conf.d/status.conf: No such file or
> > > > > > > directory
> > > > > > >
> > > > > >
> > > > > > If you're using Ubuntu, I believe it's in a different
> > > > > > location
> > > > > > -- somewhere in /etc/apache2 if memory serves.
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Tuesday, March 16, 2021, 07:20:32 PM GMT+3:30, Klaus
> > > > > > > Wenninger <kwenning at redhat.com> wrote:
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On 3/16/21 3:18 PM, Ken Gaillot wrote:
> > > > > > > > On Tue, 2021-03-16 at 09:42 +0000, Jason Long wrote:
> > > > > > > > > Hello,
> > > > > > > > > I want to launch a Clustering for my Apache Web
> > > > > > > > > Server.
> > > > > > > > > I
> > > > > > > > > have three
> > > > > > > > > servers:
> > > > > > > > >
> > > > > > > > > 1- Main server that acts as a Reverse Proxy
> > > > > > > > > 2- The secondary server that when my main server
> > > > > > > > > stopped,
> > > > > > > > > work as a
> > > > > > > > > Reverse Proxy
> > > > > > > > > 3- Apache Web Server
> > > > > > > > >
> > > > > > > > > The client ---> Reverse Proxy Server ---> Apache Web
> > > > > > > > > Server
> > > > > > > > >
> > > > > > > > > The IP addresses are:
> > > > > > > > > Main Server (node1) : 192.168.1.3
> > > > > > > > > Secondary Server (node2) : 192.168.1.10
> > > > > > > > > Apache Web Server (node3) : 192.168.1.4
> > > > > > > > >
> > > > > > > > > On the main and secondary servers, I installed and
> > > > > > > > > configured Apache
> > > > > > > > > as a Reverse Proxy Server. I created a Virtual Host
> > > > > > > > > and
> > > > > > > > > my Reverse
> > > > > > > > > Configuration is:
> > > > > > > > >
> > > > > > > > > <VirtualHost *:80>
> > > > > > > > > ProxyPreserveHost On
> > > > > > > > > ProxyPass / http://192.168.1.4/
> > > > > > > > > ProxyPassReverse / http://192.168.1.4/
> > > > > > > > > </VirtualHost>
> > > > > > > > >
> > > > > > > > > As you see, it forward all requests to the Apache Web
> > > > > > > > > Server.
> > > > > > > > >
> > > > > > > > > I installed and configured Corosync and Pacemaker as
> > > > > > > > > below:
> > > > > > > > >
> > > > > > > > > On the main and secondary servers, I opened
> > > > > > > > > "/etc/hosts"
> > > > > > > > > files and
> > > > > > > > > added my servers IP addresses and host names:
> > > > > > > > >
> > > > > > > > > 192.168.1.3 node1
> > > > > > > > > 192.168.1.10 node2
> > > > > > > > >
> > > > > > > > > Then installed Pacemaker, Corosync, and Pcs packages
> > > > > > > > > on
> > > > > > > > > both servers
> > > > > > > > > and started its service:
> > > > > > > > >
> > > > > > > > > $ sudo yum install corosync pacemaker pcs
> > > > > > > > > $ sudo systemctl enable pcsd
> > > > > > > > > $ sudo systemctl start pcsd
> > > > > > > > > $ sudo systemctl status pcsd
> > > > > > > > >
> > > > > > > > > Then Configured the firewall on both servers as
> > > > > > > > > below:
> > > > > > > > >
> > > > > > > > > $ sudo firewall-cmd --permanent --add-service=http
> > > > > > > > > $ sudo firewall-cmd --permanent --add-service=high-
> > > > > > > > > availability
> > > > > > > > > $ sudo firewall-cmd --reload
> > > > > > > > >
> > > > > > > > > After it, on both servers, I created a password for
> > > > > > > > > the
> > > > > > > > > "hacluster"
> > > > > > > > > user, then on the main server:
> > > > > > > > >
> > > > > > > > > $ sudo pcs host auth node1 node2 -u hacluster -p
> > > > > > > > > password
> > > > > > > > > node1: Authorized
> > > > > > > > > node2: Authorized
> > > > > > > > >
> > > > > > > > > Then:
> > > > > > > > > $ sudo pcs cluster setup mycluster node1 node2 --
> > > > > > > > > start
> > > > > > > > > --
> > > > > > > > > enable
> > > > > > > > > $ sudo pcs cluster enable --all
> > > > > > > > > node1: Cluster Enabled
> > > > > > > > > node2: Cluster Enabled
> > > > > > > > >
> > > > > > > > > After it:
> > > > > > > > > $ sudo pcs cluster start --all
> > > > > > > > > node1: Starting Cluster...
> > > > > > > > > node2: Starting Cluster...
> > > > > > > > >
> > > > > > > > > I checked my clusters with below command and they are
> > > > > > > > > up
> > > > > > > > > and running:
> > > > > > > > > $ sudo pcs status
> > > > > > > > > ...
> > > > > > > > > Node List:
> > > > > > > > > * Online: [ node1 node2 ]
> > > > > > > > > ....
> > > > > > > > >
> > > > > > > > > And finally, I tried to add a resource:
> > > > > > > > > $ sudo pcs resource create floating_ip
> > > > > > > > > ocf:heartbeat:IPaddr2
> > > > > > > > > ip=192.168.1.4 cidr_netmask=24 op monitor interval=5s
> > > > > > >
> > > > > > > Shouldn't the virtual-IP moved between node1 & node2 be
> > > > > > > different from the IP used for the web-server on node3?
> > > > > > > And having just one instance of the reverse-proxy that
> > > > > > > should probably be colocated with the virtual-IP - right?
> > > > > > >
> > > > > > > Klaus
> > > > > > >
> > > > > > > > > $ sudo pcs resource create http_server
> > > > > > > > > ocf:heartbeat:apache
> > > > > > > > > configfile="/etc/httpd/conf.d/VirtualHost.conf" op
> > > > > > > > > monitor
> > > > > > > > > timeout="5s" interval="5s"
> > > > > > > > >
> > > > > > > > > On both servers (Main and Secondary), pcsd service is
> > > > > > > > > enabled, but
> > > > > > > > > when I want to see my Apache Web Server then it show
> > > > > > > > > me
> > > > > > > > > below error:
> > > > > > > > >
> > > > > > > > > Proxy Error
> > > > > > > > > The proxy server received an invalid response from an
> > > > > > > > > upstream
> > > > > > > > > server.
> > > > > > > > > The proxy server could not handle the request
> > > > > > > > > Reason: Error reading from remote server
> > > > > > > > >
> > > > > > > > > Why? Which parts of my configuration is wrong?
> > > > > > > > > The output of "sudo pcs status" command is:
> > > > > > > > > https://paste.ubuntu.com/p/V9KvHKwKtC/
> > > > > > > > >
> > > > > > > > > Thank you.
> > > > > > > >
> > > > > > > > The thing to investigate is:
> > > > > > > >
> > > > > > > > Failed Resource Actions:
> > > > > > > > * http_server_start_0 on node2 'error' (1):
> > > > > > > > call=12,
> > > > > > > > status='Timed Out', exitreason='', last-rc-
> > > > > > > > change='2021-
> > > > > > > > 03-
> > > > > > > > 16 12:28:14 +03:30', queued=0ms, exec=40004ms
> > > > > > > > * http_server_start_0 on node1 'error' (1):
> > > > > > > > call=14,
> > > > > > > > status='Timed Out', exitreason='', last-rc-
> > > > > > > > change='2021-
> > > > > > > > 03-
> > > > > > > > 16 12:28:52 +03:30', queued=0ms, exec=40002ms
> > > > > > > >
> > > > > > > > The web server start timed out. Check the system,
> > > > > > > > pacemaker
> > > > > > > > and apache
> > > > > > > > logs around those times for any hints.
> > > > > > > >
> > > > > > > > Did you enable and test the status URL? The
> > > > > > > > ocf:heartbeat:apache agent
> > > > > > > > checks the status as part of its monitor (which is also
> > > > > > > > done for
> > > > > > > > start). It would be something like:
> > > > > > > >
> > > > > > > > cat <<-END >/etc/httpd/conf.d/status.conf
> > > > > > > > <Location /server-status>
> > > > > > > > SetHandler server-status
> > > > > > > > Require local
> > > > > > > > </Location>
> > > > > > > > END
> > > > > > > >
> > > > > > >
> > > > > > > _______________________________________________
> > > > > > > Manage your subscription:
> > > > > > > https://lists.clusterlabs.org/mailman/listinfo/users
> > > > > > >
> > > > > > > ClusterLabs home: https://www.clusterlabs.org/
> > > > > > >
> > > > > > > _______________________________________________
> > > > > > > Manage your subscription:
> > > > > > > https://lists.clusterlabs.org/mailman/listinfo/users
> > > > > > >
> > > > > > > ClusterLabs home: https://www.clusterlabs.org/
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Regards,
> > > > > >
> > > > > > Reid Wahl, RHCA
> > > > > > Senior Software Maintenance Engineer, Red Hat
> > > > > > CEE - Platform Support Delivery - ClusterHA
> > > > > >
> > > > > >
> > > > > > _______________________________________________
> > > > > > Manage your subscription:
> > > > > > https://lists.clusterlabs.org/mailman/listinfo/users
> > > > > >
> > > > > > ClusterLabs home: https://www.clusterlabs.org/
> > > > > > _______________________________________________
> > > > > > Manage your subscription:
> > > > > > https://lists.clusterlabs.org/mailman/listinfo/users
> > > > > >
> > > > > > ClusterLabs home: https://www.clusterlabs.org/
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Regards,
> > > > >
> > > > > Reid Wahl, RHCA
> > > > > Senior Software Maintenance Engineer, Red Hat
> > > > > CEE - Platform Support Delivery - ClusterHA
> > > > >
> > > > > _______________________________________________
> > > > > Manage your subscription:
> > > > > https://lists.clusterlabs.org/mailman/listinfo/users
> > > > >
> > > > > ClusterLabs home: https://www.clusterlabs.org/
> > > > > _______________________________________________
> > > > > Manage your subscription:
> > > > > https://lists.clusterlabs.org/mailman/listinfo/users
> > > > >
> > > > > ClusterLabs home: https://www.clusterlabs.org/
> > > > >
> > > >
> > > >
> > > > --
> > > > Regards,
> > > >
> > > > Reid Wahl, RHCA
> > > > Senior Software Maintenance Engineer, Red Hat
> > > > CEE - Platform Support Delivery - ClusterHA
> > > >
> > > > _______________________________________________
> > > > Manage your subscription:
> > > > https://lists.clusterlabs.org/mailman/listinfo/users
> > > >
> > > > ClusterLabs home: https://www.clusterlabs.org/
> > > > _______________________________________________
> > > > Manage your subscription:
> > > > https://lists.clusterlabs.org/mailman/listinfo/users
> > > >
> > > > ClusterLabs home: https://www.clusterlabs.org/
> > > >
> > >
> > >
--
Ken Gaillot <kgaillot at redhat.com>
More information about the Users
mailing list