[ClusterLabs] Off-line build-time cluster configuration

Fri Apr 24 16:36:53 EDT 2020

Thanks for the update Tomas.  Hopefully Red Hat will pick this up for a
near term release.

Craig

On Fri, Apr 24, 2020 at 1:03 AM Tomas Jelinek <tojeline at redhat.com> wrote:

> Preliminary version of this feature has been merged upstream:
>
> https://github.com/ClusterLabs/pcs/commit/425edf7aaff3c0744d9ae32bceb96f4bbfd39ee1
>
> At the moment, only RPM packages built by continuous integration system
> are available:
> https://kronosnet.org/builds/pcs/
> Note these are automatically built unofficial packages, they are not
> guaranteed to work flawlessly.
>
> Regards,
> Tomas
>
>
> Dne 16. 04. 20 v 14:00 Tomas Jelinek napsal(a):
> > Hi Craig,
> >
> > Currently, there is no support in RHEL8 for an equivalent of the --local
> > option of the 'pcs cluster setup' command from RHEL7. We were focusing
> > higher priority tasks related to supporting the new major version of
> > corosync and knet. As a part of this, the 'pcs cluster setup' command
> > has been completely overhauled providing better functionality overall,
> > like improved validations, synchronizing other files than just
> > corosync.conf and so on. Sadly, we didn't have enough capacity to
> > support the --local option in step 1.
> >
> > We are working on adding support for the --local option (or its
> > equivalent) in the near future, but we don't have any code to share yet.
> >
> >
> > Obviously, the --local version of the setup will skip some tasks done in
> > the regular cluster setup command. You are expected to do them by other
> > means. I'll put them all here for the sake of completion, even though
> > not all of them apply in your situation:
> > * check that nodes are not running or configured to run a cluster
> > * check that nodes do have cluster daemons installed in matching versions
> > * run 'pcs cluster destroy' on each node to get rid of all cluster
> > config files and be sure there are no leftovers from previously
> > configured clusters
> > * delete /var/lib/pcsd/pcs_settings.conf file (this is not done by the
> > 'pcs cluster destroy' command)
> > * distribute pcs auth tokens for the nodes
> > * distribute corosync and pacemaker authkeys, /etc/corosync/authkey and
> > /etc/pacemaker/authkey respectively
> > * synchronize pcsd certificates (only needed if you intend to use pcs
> > web UI in an HA mode)
> > * distribute corosync.conf
> > Let me know if you have any questions regarding these.
> >
> >
> > Running the current 'pcs cluster setup' command on all nodes is not
> > really an option. The command requires the nodes to be online as it
> > stores corosync.conf and other files to them over the network.
> >
> > You may, however, run it once on a live cluster to get an idea of what
> > the corosync.conf looks like and turn it into a template. I don't really
> > expect its format or schema to be changed significantly during the RHEL8
> > life cycle. I understand your concerns regarding this approach, but it
> > would give you at least some option to proceed until the --local is
> > supported in pcs.
> >
> >
> > Regards,
> > Tomas
> >
> >
> > Dne 14. 04. 20 v 20:46 Craig Johnston napsal(a):
> >> Hello,
> >>
> >> Sorry if this has already been covered, but a perusal of recent mail
> >> archives didn't turn up anything for me.
> >>
> >> We are looking for help in configuring a pacemaker/corosync cluster at
> >> the time the Linux root file system is built, or perhaps as part of a
> >> "pre-pivot" process in the initramfs of a live-CD environment.
> >>
> >> We are using the RHEL versions of the cluster products.  Current
> >> production is RHEL7 based, and we are trying to move to RHEL8.
> >>
> >> The issues we have stem from the configuration tools' expectation that
> >> they are operating on a live system, with all cluster nodes available
> >> on the network.  This is obviously not the case during a "kickstart"
> >> install and configuration process.  It's also not true in an embedded
> >> environment where all nodes are powered simultaneously and expected to
> >> become operational without any human intervention.
> >>
> >> We create the cluster configuration from a "system model", that
> >> describes the available nodes, cluster managed services, fencing
> >> agents, etc..  This model is different for each deployment, and is
> >> used as input to create a customized Linux distribution that is
> >> deployed to a set of physical hardware, virtual machines, or
> >> containers.  Each node, and it's root file system, is required to be
> >> configured and ready to go, the very first time it is ever booted.
> >> The on-media Linux file system is also immutable, and thus each boot
> >> is exactly like the previous one.
> >>
> >> Under RHEL7, we were able to use the "pcs" command to create the
> >> corosync.conf/cib.xml files for each node.
> >> e.g.
> >>            pcs cluster setup --local --enable --force --name mycluster
> >> node1 node2 node3
> >>            pcs -f ${CIB} property set startup-fencing=false
> >>            pcs -f ${CIB} resource create tftp ocf:heartbeat:Xinetd
> >>   service=tftp  --group grp_tftp
> >>            etc...
> >>
> >> Plus a little "awk" "sed" on the corosync.conf file, and we were able
> >> to create a working configuration that worked out of the box. It's not
> >> pretty, but it works in spite of the fact that we feel like we're
> >> swimming up stream.
> >>
> >> Under RHEL8 however, the "pcs cluster" command no longer has a
> >> "--local" option.  We can't find any tool to replace it's
> >> functionality.  We can use "cibadmin --empty" to create a starting
> >> cib.xml file, but there is no way to add nodes to it (or create the
> >> corosync.conf file with nodes".
> >>
> >> Granted, we could write our own tools to create template
> >> corosync.conf/cib.xml files, and "pcs -f" still works.  However, that
> >> leaves us in the unenviable position where the cluster configuration
> >> schema could change, and our tools would not be the wiser.  We'd much
> >> prefer to use a standard and maintained interface for configuring the
> >> cluster.
> >>
> >> Any suggestions would be very welcome.  While we have a non-standard
> >> use-case, we don't believe it is unrealistic given the current
> >> environment for cloud services, and automated deployment.
> >>
> >> Thanks,
> >> Craig
> >>
> >> _______________________________________________
> >> Manage your subscription:
> >> https://lists.clusterlabs.org/mailman/listinfo/users
> >>
> >> ClusterLabs home: https://www.clusterlabs.org/
> >>
> >
> > _______________________________________________
> > Manage your subscription:
> > https://lists.clusterlabs.org/mailman/listinfo/users
> >
> > ClusterLabs home: https://www.clusterlabs.org/
>
> _______________________________________________
> Manage your subscription:
> https://lists.clusterlabs.org/mailman/listinfo/users
>
> ClusterLabs home: https://www.clusterlabs.org/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.clusterlabs.org/pipermail/users/attachments/20200424/6a282b01/attachment.htm>