WIP: erp5: Introduce mariadb replication at SlapOS level
1.
Allow a mariadb to act as a replica by requesting it with parameters:
"replication": {
"bootstrap-url": <some-url-that-statically-serves-a-recent-dump>
"primary-url": <mysql://replication_user:password@ip:port>
"seconds-behind-master-threshold": <Monitoring threshold for replication promise, optional, defaults to 0>
}
Such mariadb will go into replica mode on initialization of the database: this means changing an existing mariadb instance to add these parameters will have no effect (almost*); existing data is never deleted and mariadb goes into replica mode only if there is no data yet.
*almost: a mariadb requested with "replication"
parameters results in mariadb_update
service being disabled because it interferes with replication - even if due to pre-existing data the mariadb is not actually in replica mode. This is not ideal because it means that a mariadb primary that is requested with replication
parameters There is talk to eventually remove this service entirely and implement its functionality another way.
Note: The bootstrap-url
parameter is used to bootstrap the replication process by first restoring the replica mariadb to a recent backup of the primary database. This is usually necessary because only the last few days of binlogs are kept on the primary, so the replica must first be bootstrapped to a state more recent that the oldest available binlog before it can start replicating. This bootstrap-url
can be skipped (only) when the primary is recent enough that all binlogs since the creation of the primary are still available. If the primary has just been created, being able to skip bootstrapping is useful because the primary might not even have a first backup yet.
TODO:
-
Allow a replica mariadb to stop replicating and become a primary without requiring manual login to the instance and manual operations on the DB (e.g. by providing a url where the user can click to perform this action). This will be a necessary step of an eventual automated takeover procedure.
-
Find a better solution for
mariadb_update
functionality.
2.
Make any mariadb (replica or primary) a) statically serve recent backups (dumps) on the same IP as the mariadb server and b) have a configured replication_user
with random password, and publish two corresponding connection parameters replication-bootstrap-url
and replication-primary-url
, to be used to setup a replica mariadb.
Note: This is currently insecure when using public IPv6, but it can already be used on private IPv4 when the primary and the replica are in the same LAN.
TODO:
-
Use SSL on public IP: serve the backups with SSL and proxy the mariadb server with SSL on public IP (instead of enabling SSL in mariadb directly, to allow SSL-less access with private IP (?)). This will also impact the replica-initialisation logic. Ideally, use (mutual?) authentication with trusted certificates (provided by caucase?). As a temporary step, maybe self-signed SSL with passwords published in connection parameters?
-
Use mariabackup instead of dumps to allow fast bootstrapping of a replica. This will affect the replica-initialisation logic as well.
-
Propagate these mariadb connection parameters in erp5 root instance.
-->mariabd-replication-primary-url
andmariadb-replication-bootstrap-url
3.
Include some miscellaneous fixes for mariadb-with-IPv6 and gcc-version-for-Python2-SRs.