[ClusterLabs] Resource(s) for mysql+galera cluster

[lejeczek]

On 15/12/2021 08:16, Michele Baldessari wrote:
> Hi,
>
> On Tue, Dec 14, 2021 at 03:09:56PM +0000, lejeczek via Users wrote:
>> I failed to find any good or any for that matter info, on present-day
>> mariadb/mysql galera cluster setups - would you know of any docs which
>> discuss such scenario comprehensively?
>> I see there among resources are 'mariadb' and 'galera' but which one to use
>> I'm still confused.
> in the deployment of OpenStack via TripleO[1] we set up galera in
> multi-master mode using the galera resource agent[2] (it runs in bundles
> aka containers in our specific case, but that is tangential).
>
> Here is an example of the pcs commands we use to start it (these are
> a bit old, things might have changed since):
>
> pcs property set --node controller-1 galera-role=true
> pcs resource bundle create galera-bundle container docker image=192.168.24.1:8787/rhosp13/openstack-mariadb:pcmklatest replicas=3 masters=3 options="--user=root --log-driver=journald -e KOLLA_CONFIG_STRATEGY=COPY_ALWAYS" run-command="/bin/bash /usr/local/bin/kolla_start" network=host storage-map id=mysql-cfg-files source-dir=/var/lib/kolla/config_files/mysql.json target-dir=/var/lib/kolla/config_files/config.json options=ro storage-map id=mysql-cfg-data source-dir=/var/lib/config-data/puppet-generated/mysql/ target-dir=/var/lib/kolla/config_files/src options=ro storage-map id=mysql-hosts source-dir=/etc/hosts target-dir=/etc/hosts options=ro storage-map id=mysql-localtime source-dir=/etc/localtime target-dir=/etc/localtime options=ro storage-map id=mysql-lib source-dir=/var/lib/mysql target-dir=/var/lib/mysql options=rw storage-map id=mysql-log-mariadb source-dir=/var/log/mariadb target-dir=/var/log/mariadb options=rw storage-map id=mysql-log source-dir=/var/log/containers/mysql target-dir=/var/log/mysql options=rw storage-map id=mysql-dev-log source-dir=/dev/log target-dir=/dev/log options=rw network control-port=3123 --disabled
>
> pcs constraint location galera-bundle rule resource-discovery=exclusive score=0 galera-role eq true
>
> pcs resource create galera ocf:heartbeat:galera log='/var/log/mysql/mysqld.log' additional_parameters='--open-files-limit=16384' enable_creation=true wsrep_cluster_address='gcomm://controller-0.internalapi.localdomain,controller-1.internalapi.localdomain,controller-2.internalapi.localdomain' cluster_host_map='controller-0:controller-0.internalapi.localdomain;controller-1:controller-1.internalapi.localdomain;controller-2:controller-2.internalapi.localdomain' meta master-max=3 ordered=true container-attribute-target=host op promote timeout=300s on-fail=block bundle galera-bundle
>
> I don't think we have any docs discussing tradeoffs around this setup,
> but maybe this is a start. If you want multi-master galera then you can
> do it with the galera RA. If you need to look at more recent configs
> check out [3] (Click on a job and the logs and you can check something
> like [4] for the full pcs config)
>
> hth,
> Michele
>
> [1] https://docs.openstack.org/tripleo-docs/latest/
> [2] https://github.com/ClusterLabs/resource-agents/blob/master/heartbeat/galera.in
> [3] https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-master
> [4] https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-master/31594c2/logs/overcloud-controller-0/var/log/extra/pcs.txt.gz
I had used 'galera' OCF resource which worked, following man 
page for it but if I can share an opinion - it's seems that 
most(if not all developers) "suffer" from the same problem - 
namely, inability to write a solid, very good man/docs 
pages, which is the case with HA/Pacemaker. (there are 
exceptions such as 'lvm2', 'systemd', 'firewalld' & more 
(I'm a CentOS/Fedora user))

Especially man pages suffer - while developers know that 
it's first place user/admin goes to.

So I struggle with multi-master setup - many thanks for the 
examples - "regular" master/slave works, but that gives out 
only one node having 'mariadb' up&running, whereas remaining 
nodes keep resource in 'standby' mode - as I understand it.
When I try to fiddle with similar to you configs then 
'galera' tells me:
..
Galera must be configured as a multistate Master/Slave resource.
..
and my setup should be simpler - no containers, no bundles - 
only 3 nodes.

many thanks, L